Incomplete Post, this contains only src-nat part, second method of NETMAP will be added soon which is far more simple & efficient as compared to the src-nat method
My humble request, Kindly donot consider me as an expert on this stuff, I am NOT certified in anything Mikrotik/Cisco/Linux or Windows. However I have worked with some networks and I read , research & try stuff all of the time. When you are enslaved by private job & working as one man army, you have to perform many task in which you are not formally trained for. So I am not speaking/posting about stuff I am formerly trained in, I pretty much go with experience and what I have learned on my own. And , If I don’t know something then I read & learn all about it.
So , please don’t hold me/my-postings to be always 100 percent correct. I make mistakes just like everybody else. However – I do my best, learn from my mistakes and try to help others
Some references used in this post
https://mum.mikrotik.com/presentations/EU18/presentation_5195_1524667160.pdf
https://wiki.mikrotik.com/wiki/Manual:IP/Firewall/NAT#Carrier-Grade_NAT_.28CGNAT.29_or_NAT444
Scenario:
OP is running mini ISP with around 200 active subscribers. Mikrotik Router is being used as PPPoE Server along with Freeradius as AAA. On Mikrotik, one public IP is configured for WAN and additional /24 routed pool (256 public IP addresses) is provided to the OP via ISP so that he can provide public IP to each user. After the network upgrades , OP have reached 700 users in total, and since he have only 256 public ip’s , he is now using natting for half of his users.
We all know that IPV4 shortage is on peak , getting ipv4 is expensive for 3rd world countries & small ISP’s as well.
This NATTING workaround is creating hurdles in tracking illegal activity performed by any NATTED users because hundreds of NATTED user will have same public ip (Mikrotik WAN IP). nowadays law sometimes provide only the public ip along with source port and ask for the user credentials details for investigation purposes.
with single public IP and hundreds of natted hosts behind it. tracking is nearly impossible.
CG-NAT as Workaround:
CGNAT concept is used to share one or preferably more public IP addresses with large number of private ip addresses on ratio basis.CGNAT/NAT444 is a conception, not a function. In terms of RouterOS functionality it’s simple SRC NAT rule.
To combat with this IPV4 exhausting issue, we can use CGNAT as a workaround. This is by no means a solution, & the OP should get public IP space (either ipv4 or ipv6) to comply with the LAW.
in my personal experience , Deployment is very hectic, tracking is very daunting task ! z@ib
Hardware/Software Used in this post:
- Mikrotik CCR1036 Routerboard
- IBM Xseries 3650 M5 , Xeon 8 Cores CPU x 2 / 64 GB RAM / 600GB 10k rpm SAS x 16 Disks in RAID10 mode for fast read/write access, Vmware ESXI 6.7u3 installed
- VM#1 Ubuntu 18.4 with Freeradius 3.20 for AAA
- VM#2 Ubuntu 16.4 as SYSLOG-NG for LOGS storage of Mikrotik & other devices on the network like Cisco switches / barracuda / etc
IP scheme example used in this post:
Public IP range: (/24 public IP’s routed pool)
- 1.1.1-1.1.1.255
- Total Public IP useable: 255
Private IP range for PPPoE users:
- 172.16.1.1-172.16.1.255
- 172.16.2.1-172.16.2.255
- 172.16.3.1-172.16.3.255
- Total Private IP useable: 765
For 765 Users, we will be using 1:5 Ratio, thus 153 public ips will be used for 765 users. (on a ratio of 1:5).
- per private IP, we will reserve 10,000 ports, which should be more than enough for each user.
- per private IP, we will be creating 3 rules, one for TCP, second for UDP, 3rd for non ports range
CGNAT configuration on RouterOS is very much similar to regular source NAT configuration.
Some possible disadvantages of using CGNAT concept:
- CGNAT is not sustainable in the long term, hectic to manage the private/public pools specially if you have multiple NASes doing same job
- For tracking, you should use script “on-logon” and “on-logout” and send the logs to a Remote Syslog server, if freeradius is being used, then it will handle this part
- ISP deploying IP address sharing techniques should also deploy a corresponding logging architecture to maintain records of the relation between a customer’s identity and IP/port resources utilized
- You should deploy additional SYSLOG server (either windows or linux base) to store logs. I would prefer linux base SYSLOG-NG). Tracking of users for legal reasons means searching hundreds GB’s of logging would be required, as multiple end users go behind one (or more) public IP address(es)
- Tracking Logs is not an easy task particularly when you have tongs of Logging (in a DB).Logging every NAT translation is resource consuming. Some super fast computing resources (including preferably RAID10 or SSD based storage) and fine tune DB would be required
-
A CG-NAT device must use the same external IP address mappingfor all sessions associated with the same internal IP address
-
Preserve Range RFC4787 defines two port ranges: “Well Known Ports” [0, 1023] and “Registered“/”Dynamic and/or Private” [1024, 65535] When the source port of the internal host establishing a new connection falls into one of these ranges the CGN tries to allocate an external source port in the same range. If it fails to find a port, connection fails too.
- Most Applications do not behave well with TCP resets
-
Many operators are still not familiar with CG-NAT complexities. There is a lot of trial and error on the part of ISP’s
To add multiple Public IP addresses on WAN interface in bulk using single CMD on Terminal
You may need to add all of your public IP addresses (which will be used for CGNAT) on WAN interface. to do this in bulk you can use Mikrotik FOR X script function for ease / ZAIB
:for x from 1 to 153 do={ /ip address add address="1.1.1.$x/32" comment="1.1.1.$x - Routed IP for ppp CGNAT - zaib" interface="ether1-wan"}
Adding FUNCTION in Mikrotik for later Automation
Paste this in Mikrotik RouterOS terminal:
# CGNAT script source: https://wiki.mikrotik.com/wiki/Manual:IP/Firewall/NAT#Carrier-Grade_NAT_.28CGNAT.29_or_NAT444
# I modified this script script to match local setup requirements and to support latest version of ROS 6.46.1 as of December 2019. So its somewhat different from the original
# Syed Jahanzaib / aacable at hotmail dot com
# I had to add this global sqrt in order to execute the script on latest ros
:global sqrt
:global sqrt do={
:for i from=0 to=$1 do={
:if (i * i > $1) do={ :return ($i - 1) }
}
}
:global addNatRules do={
/ip firewall nat add chain=srcnat action=jump jump-target=xxx \
src-address="$($srcStart)-$($srcStart + $count - 1)"
:local x [$sqrt $count]
:local y $x
:if ($x * $x = $count) do={ :set y ($x + 1) }
:for i from=0 to=$x do={
/ip firewall nat add chain=xxx action=jump jump-target="xxx-$($i)" \
src-address="$($srcStart + ($x * $i))-$($srcStart + ($x * ($i + 1) - 1))"
}
:for i from=0 to=($count - 1) do={
:local prange "$($portStart + ($i * $portsPerAddr))-$($portStart + (($i + 1) * $portsPerAddr) - 1)"
# following is modified too
/ip firewall nat add chain="xxx-$($i / $x)" action=src-nat protocol=tcp src-address=($srcStart + $i) \
to-address=$toAddr to-ports=$prange
/ip firewall nat add chain="xxx-$($i / $x)" action=src-nat protocol=udp src-address=($srcStart + $i) \
to-address=$toAddr to-ports=$prange
/ip firewall nat add chain="xxx-$($i / $x)" action=src-nat src-address=($srcStart + $i) to-address=$toAddr
}
}
Now we can issue following CMD to add rules in NAT section
# per private IP, we will reserve 10000 ports, which should be more than enough for each user. # per private IP, we will be creating 3 rules, one for TCP, second for UDP, 3rd for non ports range $addNatRules count=5 srcStart=172.16.1.1 toAddr=1.1.1.1 portStart=10000 portsPerAddr=9999 $addNatRules count=5 srcStart=172.16.1.6 toAddr=1.1.1.2 portStart=10000 portsPerAddr=9999 $addNatRules count=5 srcStart=172.16.1.11 toAddr=1.1.1.3 portStart=10000 portsPerAddr=9999 $addNatRules count=5 srcStart=172.16.1.16 toAddr=1.1.1.4 portStart=10000 portsPerAddr=9999 $addNatRules count=5 srcStart=172.16.1.21 toAddr=1.1.1.5 portStart=10000 portsPerAddr=9999 $addNatRules count=5 srcStart=172.16.1.26 toAddr=1.1.1.6 portStart=10000 portsPerAddr=9999 $addNatRules count=5 srcStart=172.16.1.31 toAddr=1.1.1.7 portStart=10000 portsPerAddr=9999 $addNatRules count=5 srcStart=172.16.1.36 toAddr=1.1.1.8 portStart=10000 portsPerAddr=9999 $addNatRules count=5 srcStart=172.16.1.41 toAddr=1.1.1.9 portStart=10000 portsPerAddr=9999 $addNatRules count=5 srcStart=172.16.1.46 toAddr=1.1.1.10 portStart=10000 portsPerAddr=9999 # & so on for rest of the pool, you can further automate this by using additional functions & scripting
Enable Logging of CG-NAT Output:
# To log user IP/NAT information on LOG window / files or remote syslog /ip firewall mangle add action=accept chain=prerouting connection-state=new log=yes log-prefix="NAT_INFO_F> " src-address=172.16.0.0/16
Log Result (from different servers , so ip scheme may be changed in these logs, just for example purposes)
In this log you can clearly see the src-dst address, and on which public ip request was natted along with ports. This is useful
Jan 1 16:58:21 10.11.11.36 firewall,info NAT_INFO_F> forward: in:ether10 out:ether1, src-mac b4:2e:99:xx:xx:xx, proto UDP, 10.11.11.161:63179->172.217.19.3:443, NAT (101.11.11.161:63179->1.1.1.1:19082)->172.217.19.3:443, len 428 Jan 1 16:58:21 10.11.11.36 firewall,info NAT_INFO_F> forward: in:ether10 out:ether1, src-mac b4:2e:99:xx:xx:xx, proto UDP, 10.11.11.161:63179->172.217.19.3:443, NAT (10.11.11.161:63179->1.1.1.1:19082)->172.217.19.3:443, len 428
another one:
To delete older logs from syslog mysql db,
mysql -uroot -pSQLPASSWORD -s -e "use syslog; DELETE FROM logs WHERE date(datetime) < (CURDATE() - INTERVAL 3 MONTH);"