Quantcast
Channel: Syed Jahanzaib – Personal Blog to Share Knowledge !
Viewing all 409 articles
Browse latest View live

Mikrotik with Freeradius/mySQL – Trimming & Archiving RADACCT # Part-8

$
0
0

fre


FREERADIUS WITH MIKROTIK – Part #1 – General Tip’s>

FREERADIUS WITH MIKROTIK – Part #2  – COA

FREERADIUS WITH MIKROTIK – Part #3 – Expiration

FREERADIUS WITH MIKROTIK – Part #4 – Auto Mac Binding

FREERADIUS WITH MIKROTIK – Part #5 – Stale Sessions

FREERADIUS  WITH MIKROTIK – Part # 6 – External Auth Script & RADPOSTAUTH

FREERADIUS WITH MIKROTIK – Part #7  – Quota Limit

FREERADIUS WITH MIKROTIK – Part #8  – RADACCT – Trimming & Archiving < You are here


Personnel Note:

This is another post about freeradius. My aim is to let people know that creating your own Radius Billing system is not an ROCKET SCIENCE. The only thing required is the ultimate passion to achieve the goal. And with the right search, reading, understanding logic’s, you can do all on your own. I strongly encourage to read the FR mailing list and Google

OS: Ubuntu 16.04.3 LTS / 64bit


Disclaimer:
There are some other neat-to-Perfect methods like archive users usage data from radacct table to the archive data , per user ONE row only.  This will take much lesser size , but I need to track some data therefore I duplicated the whole table. Else you can use TRIGGER on RADACCT too which can update other user table for the usage only. All depend son the scenario and requirements.



Scenario:

slow data.jpg

Radius accounting is stored in radacct table which contains the user usage related data . Over the period of time this data can take lot of space as time passes. It can also create performance related issues as well if you donot take care of it like slow or timeout queries because of hundreds of thousands of entries.

I received complains from few networks that radius is giving timeout because they were installed few years ago and there radacct table have grown enormously in size thus resulting ins slow queries and timeout in authentication specially in Radius Manager which uses external auth script to validate the user request.

Solution:

  • Use SSD disks (or RAID10), they are more performance oriented storage,
  • Adding more RAM is plus point for any database server.

We will create another table named  radacct_archive  and move accounting data older then 6 months from the radacct table to radacct_archive table.

This way queries will work faster dueto less data in radacct. Also we will delete data older then 12 months from `radacct_archive` table too so that it may not grow large later.

First we will clone the radacct table structure into new radacct_archive table.

# This is one time step.
mysql -uroot -pzaib1234 -s -e "use radius; create table radacct_archive LIKE radacct;"

Now you can create a bash script and schedule it to run DAILY in less busy timings like 5 am in the morning.

# --- Copy data from CURRENT radacct table to new radacct_archive table (for archive purposes)
mysql -uroot -pSQLPASS -s -e "use radius; INSERT INTO radacct_archive SELECT * FROM radacct WHERE acctstoptime > 0 AND date(acctstarttime) < (CURDATE() - INTERVAL 6 MONTH);"

# --- Now Delete data from CURRENT RADACCT table so that it should remain fit and smart ins size
mysql -uroot -pSQLPASS -s -e "use radius; DELETE FROM radacct WHERE acctstoptime > 0 AND date(acctstarttime) < (CURDATE() - INTERVAL 6 MONTH);"

# --- Now Delete data from RADACCT_ARCHIVE table so that it should not grow either more then we required i.e 1 Year - one year archived data is enough IMO
mysql -uroot -pSQLPASS -s -e "use radius; DELETE FROM radacct_archive WHERE date(acctstarttime) < (CURDATE() - INTERVAL 12 MONTH);"

Regard’s
Syed Jahanzaib


FREERADIUS WITH MIKROTIK – Part #9 – Restrict user auth from allowed VLAN only

$
0
0

FreeRadius Core info

restrcited vlan.jpg

FREERADIUS WITH MIKROTIK – Part #1 – General Tip’s>

Disclaimer:
This solution was designed for some specific situation described below. Surely there might be better ways, but sometimes dueto my laziness & dumbness, I select whichever gives required results. You may fine tune it, or modify it as required. I always recommends to read the FR mailing list for any query.

It’s a weird world we live in.
EVERY HUMAN BEING IS BORN FREE BUT YET EVERYWHERE HE IS IN CHAINS … ~

Syed Jahanzaib ~


Scenario:

OP have single Mikrotik CCR as NAS (pppoe server) and Freeradius as billing system. VLAN(s) are configured on Cisco switch for each reseller/dealer area with proper Trunking configured from SW to CCR. OP have few reseller/franchise managers, like Dealer-A, Dealer-B. They can create there own users in freeradius using customized fronted designed in php. All dealers can view/add/edit there own users only.

Sometimes it happens that Dealer-A creates User-ID and provide it to a user/friend who is sitting in Dealer-B network, therefore from Billing perspective its a Loss for Dealer-B.


Requirement:

To prevent users roaming. Every user must be allowed to connect from there respective dealer network Only. [That is ‘allowed-vlan]. User’s MAC address validation is also in place but we know that mac can be cloned easily.


Solution:

p4.jpg

We have a user table which contains a column vlanid. We will add a SQL IF statement which will check the the connecting user NAS-Port-Id & match it with users allowed vlanid in the user’s table.

Users Table Example:

mysql> describe users;
+-------------------+--------------+------+-----+-------------------+-----------------------------+
| Field | Type | Null | Key | Default | Extra |
+-------------------+--------------+------+-----+-------------------+-----------------------------+
| id | int(10) | NO | PRI | NULL | auto_increment |
| username | varchar(128) | NO | UNI | NULL | |
| password | varchar(32) | NO | | NULL | |
| firstname | text | NO | | NULL | |
| lastname | text | NO | | NULL | |
| email | text | NO | | NULL | |
| mobile | text | NO | | NULL | |
| cnic | text | NO | | NULL | |
| srvname | text | NO | | NULL | |
| srvid | int(3) | NO | | NULL | |
| expiration | date | YES | | NULL | |
| mac | varchar(30) | NO | | NULL | |
| bwpkg | varchar(256) | NO | | NULL | |
| pool | varchar(128) | YES | | other | |
| is_enabled | int(1) | NO | | NULL | |
| is_days_expired | int(1) | NO | | NULL | |
| is_qt_expired | int(1) | NO | | NULL | |
| is_uptime_expired | int(1) | NO | | NULL | |
| qt_total | varchar(32) | NO | | NULL | |
| qt_used | varchar(20) | NO | | NULL | |
| uptime_limit | varchar(20) | NO | | NULL | |
| uptime_used | varchar(32) | NO | | NULL | |
| owner | text | NO | | NULL | |
| vlanid | varchar(32) | NO | | NULL | |
| createdon | timestamp | NO | | CURRENT_TIMESTAMP | on update CURRENT_TIMESTAMP |
+-------------------+--------------+------+-----+-------------------+-----------------------------+

mysql> select * from users;
+----+----------+----------+----------------+---------------+---------------------+-------------+-------------------+---------+-------+------------+-------------------+-------------+----------------------+------------+-----------------+---------------+-------------------+----------+---------+--------------+-------------+--------+-------------+---------------------+
| id | username | password | firstname | lastname | email | mobile | cnic | srvname | srvid | expiration | mac | bwpkg | pool | is_enabled | is_days_expired | is_qt_expired | is_uptime_expired | qt_total | qt_used | uptime_limit | uptime_used | owner | vlanid | createdon |
+----+----------+----------+----------------+---------------+---------------------+-------------+-------------------+---------+-------+------------+-------------------+-------------+----------------------+------------+-----------------+---------------+-------------------+----------+---------+--------------+-------------+--------+-------------+---------------------+
| 1 | zaib | zaib | OK | jahanzaib | aacableAThotmail.com | 03333021909 | 1234567890-1-1 | | 0 | 2018-01-04 | 00:0C:29:B9:D8:A0 | 1024k/1024k | public-pool | 1 | 0 | 0 | 0 | 300 | 38980 | | | manager1 | ether1-LAN | 2018-01-17 16:45:05 |

Now we will add the SQL IF statement that will actually check every incoming Authentication request for matching vlan with users table.

Edit Default Sites-Enabled file,

nano /etc/freeradius/sites-enabled/default

& paste following in `Authorize` Section

if ("%{sql: select vlanid from users where username = '%{User-Name}'}" != "%{NAS-Port-Id}") {
update reply {
Reply-Message = 'Error: You are not allowed to connect form this VLAN !'
}
update control {
Auth-Type := "Reject"
}
}

Save & Exit.

Now reload Freeradius in Debug Mode (by freeradius -X) & monitor the debugging.

If user will connect from another vlan which does not matches the vlanid in user’s table he will get denied with the message.

Note: For testing purposes . I have added dummy entry in user’s vlanid.

rad_recv: Access-Request packet from host 101.11.11.253 port 34175, id=11, length=119
Service-Type = Framed-User
Framed-Protocol = PPP
NAS-Port = 14
NAS-Port-Type = Ethernet
User-Name = "zaib"
Calling-Station-Id = "00:0C:29:B9:D8:A0"
Called-Station-Id = "testppp"
NAS-Port-Id = "ether1-LAN"  # < < < <  Look at this, this is the VLAN id user is coming from, this will be matched with the vlanid we add in the users table
User-Password = "zaib"
NAS-Identifier = "Win"
NAS-IP-Address = 101.11.11.253
# Executing section authorize from file /etc/freeradius/sites-enabled/default
+group authorize {
++[preprocess] = ok

++? if ("%{sql: select vlanid from users where username = '%{User-Name}'}" != "%{NAS-Port-Id}")
sql_xlat
expand: %{User-Name} -> zaib
sql_set_user escaped user --> 'zaib'
expand: select vlanid from users where username = '%{User-Name}' -> select vlanid from users where username = 'zaib'
rlm_sql (sql): Reserving sql socket id: 29
sql_xlat finished
rlm_sql (sql): Released sql socket id: 29
expand: %{sql: select vlanid from users where username = '%{User-Name}'} -> ether1-LAN-DUMMY
expand: %{NAS-Port-Id} -> ether1-LAN
? Evaluating ("%{sql: select vlanid from users where username = '%{User-Name}'}" != "%{NAS-Port-Id}") -> TRUE
++? if ("%{sql: select vlanid from users where username = '%{User-Name}'}" != "%{NAS-Port-Id}") -> TRUE
++if ("%{sql: select vlanid from users where username = '%{User-Name}'}" != "%{NAS-Port-Id}") {
+++update reply {
+++} # update reply = noop
+++update control {
+++} # update control = noop
++} # if ("%{sql: select vlanid from users where username = '%{User-Name}'}" != "%{NAS-Port-Id}") = noop

Sending delayed reject for request 0
Sending Access-Reject of id 11 to 101.11.11.253 port 34175

# Meaningful Reply Message by zaib
Reply-Message = "Error: You are not allowed to connect form this VLAN !"

Waking up in 4.9 seconds.
Cleaning up request 0 ID 11 with timestamp +31
Ready to process requests.

.

& if the users request matches , he will be granted access (off course after all other checks):)


 

p4-2.jpg

Manipulating Date Functions in Mikrotik

$
0
0

.

Scenario:

In Mikrotik router , we have a script which is scheduled to run every 5 minutes. It gets in+out bytes for specific interface usage (using firewall counters) and save/add in a file,  when date changes, it simply calculate the total usage and sends report via email to the admin. it’s pretty handy and quick way to get your wan usage with customization. Example of email report via script:

mail for wan total

Problem:

if the script is sending email to the admin about whole day usage at 12:05 am , it will send the current date, whereas the usage is for yesterday. therefore we want to display yesterday day date in the subject. like Current date - 1 day

Solution:

The below script along with separate supporting function script, can manipulate date function in mikrotik. Therefore using this script function we can get yesterday date in our email subject to show correct date.

Surely there must be more simpler methods that I am really unaware of I used this method and it works fine. in Linux date manipulation is way too easy example

TODAY=date -d "yesterday" '+%Y-%m-%d'
But in Mikrotik we dont have such easy functions.

Ok moving forward… We need to make one main function script that can be called by any other script to get the required date manipulation.This will be one time script & will function only if its called from another script.

Script# – For date manipulation function
Name: func_date

# This script is for Mikrotik ROS to manipulate date functions as per requirements , you can modify it as required
# This function script is copied from following mikrotik forum. please see
# https://forum.mikrotik.com/viewtopic.php?f=9&t=127050&p=625209#p625209

# Syed Jahanzaib

:local mdays {31;28;31;30;31;30;31;31;30;31;30;31}
:local months {"jan"=1;"feb"=2;"mar"=3;"apr"=4;"may"=5;"jun"=6;"jul"=7;"aug"=8;"sep"=9;"oct"=10;"nov"=11;"dec"=12}
:local monthr {"jan";"feb";"mar";"apr";"may";"jun";"jul";"aug";"sep";"oct";"nov";"dec"}
:local dd [:tonum [:pick $date 4 6]]
:local yy [:tonum [:pick $date 7 11]]
:local month [:pick $date 0 3]
:local mm (:$months->$month)
:set dd ($dd+$days)
:local dm [:pick $mdays ($mm-1)]
:if ($mm=2 && (($yy&3=0 && ($yy/100*100 != $yy)) || $yy/400*400=$yy) ) do={ :set dm 29 }
:while ($dd>$dm) do={
:set dd ($dd-$dm)
:set mm ($mm+1)
:if ($mm>12) do={
:set mm 1
:set yy ($yy+1)
}
:set dm [:pick $mdays ($mm-1)]
:if ($mm=2 && (($yy&3=0 && ($yy/100*100 != $yy)) || $yy/400*400=$yy) ) do={ :set dm 29 }
};
:local res "$[:pick $monthr ($mm-1)]/"
:if ($dd<10) do={ :set res ($res."0") }
:set $res "$res$dd/$yy"
:return $res

Script#2 – Our script that requires Yesterday Date
Name: test

In this example we will get YESTERDAY date.

# You can change the DAYS (-1) as per your requirements like previous or ahead using +)

:local shiftDate [:parse [/system script get func_date source]]
:local DT ([/system clock get date])
:local LASTDAY [$shiftDate date=$DT days=-1]
:put "TODAY Date = $DT"
:put "YESTERDAY date = $LASTDAY"

DRY RUN!

Now run the test script which will result in yesterday date, as showed below …

[zaib@CCR_GW] > /sys scr run test
# OUTPUT RESULT

TODAY Date = jan/23/2018
YESTERDAY date = jan/22/2018

Now you can use such function in any script as per your own requirements…

Regard’s
Syed Jahanzaib

 

FREERADIUS WITH MIKROTIK – Part #10 – Allow/Deny User Login by Time

$
0
0

Stop-watch

FREERADIUS WITH MIKROTIK – Part #1 – General Tip’s Click here to read more on FR tutorials …


Task:

Allow specific Users login in specific timings only using Login-Time module in freeradius using GROUPS rather than single entry for each user in radcheck table.

The requirement was raised by an OP whose bandwidth was idle in late night timings, therefore they wanted to introduce night packages for the sake of user attraction. In this guide i used morning time instead just for an example.

Example:

user zaib is a member of 8am8pm group and this groups should be able to login from 8am till 8pm all week.


Solution:

First make sure login modules is added in AUTHORIZE section in sites-enabled/default file.

login module

 

TIP: in below section GROUP base approach is being used rather then individual user in radcheck section. To better understand what group does what, you better read this.

https://www.serveradminblog.com/2011/12/freeradius-install-howto-4-populating-tables/

Now add the user/group related entries in the appropriate tables. Like

Username entry
mysql> select * from radcheck;
+----+----------+----------------------+----+----------------------+
| id | username | attribute | op | value |
+----+----------+----------------------+----+----------------------+
| 1 | zaib | Cleartext-Password | := | zaib |
+----+----------+----------------------+----+----------------------+
Add Group Check
mysql> select * from radgroupcheck;
+----+-----------+------------+----+-------------+
| id | groupname | attribute | op | value |
+----+-----------+------------+----+-------------+
| 1 | 8am8pm | Login-Time | := | Al0800-2000 |
+----+-----------+------------+----+-------------+
Add Group Reply
mysql> select * from radgroupreply;
+----+-----------+---------------------+----+--------------+
| id | groupname | attribute | op | value |
+----+-----------+---------------------+----+--------------+
| 17 | 8am8pm | Login-Time | := | Al0800-2000 |
+----+-----------+---------------------+----+--------------+
Add User in 8am8pm group
mysql> select * from radusergroup;
+----+----------+-----------+----------+
| id | username | groupname | priority |
+----+----------+-----------+----------+
| 3 | zaib | 8am8pm | 1 |
+----+----------+-----------+----------+

Testing ….

Issue radclient command to see the reply from FR…

echo "User-Name = zaib, Password = zaib, Calling-Station-Id =00:0C:29:35:F8:2F" | radclient -s localhost:1812 auth testing123

OUTPUT:

Received response ID 195, code 3, length = 89

Reply-Message = "REJECT Reason: Calling Outside allowed timespan (Al0800-2000), zaib"

Total approved auths: 0
Total denied auths: 1
Total lost auths: 0

Done.

Regard’s
Syed Jahanzaib

 

Mikrotik: Schedule script to run in specific day(s) of week only

$
0
0


Scenario:

In mikrotik, we want to execute a script which should run on following days at given time. Monday to Friday , 8am Sounds easy enough! but unfortunately Mikrotik doesn’t provides flexible scheduler like windows which let you select days by just clicking. Therefore we have to create some smart function that calculate current day and pass the variable to action part in the script, which than compare variable with its specific days like mon-fri & if it matches then take action ELSE goto Sleep 🙂

Regard's
Syed Jahanzaib

Solution!

Script!

# Script to check current day name, and take action accordingly ONLY if matched with our defined days
# Example disable specific wan link only which is a quota base service but free in specific timings
# Made it for an OP who was using NAYATEL WAN links which are quota base.
# But good part is they provide free airtime (no quota count) from 8pm till 8am Daily & on saturday + sunday its total free as well
# Therefore OP wants to disable the link only from mondy to friday at 8am, but it should not disable if its saturday or sunday.
# unfortuantely In mikrotik we dont have option for selecting DAYS like in windows task scheudler, pitty ...
# This is why we will use 'work smarter, not harder' approach for this particular case.
# This script will send EMAIL using GMAIL account & SMS using local KANNEL base SMS server (installed locally with GSM modem with mobile SIM)
# Syed Jahanzaib / aacable [at] hotmail [for] com / 27-January-2018

# The script may contain Junk data as well, please trim it before using
# Let's START....
# Decalring Variable..

# ISP NAME
:local ISP "NAYATEL"
# Finding day of week function is by melboyscout (melboyscout [at] gmail.com)
:local date [/system clock get date]
:local result ""
:local months [:toarray "jan,feb,mar,apr,may,jun,jul,aug,sep,oct,nov,dec"]
:local daytbl [:toarray "sun,mon,tue,wed,thu,fri,sat"]
:local month [:pick $date 0 3]
:local day [:pick $date 4 6]
:local dayc [:pick $date 5 6]
:local year [:pick $date 7 11]
# if the first char is a 0 (zero) only read last char, else script fails
:if ([:pick $date 4 5] = 0) do={ :set day ($dayc)}
:local sum 0
:local aaa 0
:local yyy 0
:local mmm 0
:local nmonth 1
:for mindex from=0 to=[:len $months] do={
:if ([:pick $months $mindex] = $month) do={:set nmonth ($mindex + 1) }
}
:set aaa ((14 - $nmonth) / 12)
:set yyy ($year - $aaa)
:set mmm ($nmonth + 12 * $aaa - 2)
:set sum (7000 + $day + $yyy + ($yyy / 4) - ($yyy / 100) + ($yyy / 400) + ((31 * $mmm) / 12))
:set sum ($sum - (($sum / 7) * 7))
:set result [:pick $daytbl $sum]
:put $result
# END Math Calculation

# Start IF matching to see if current day is what we required, if yes take action
# Example we want to run scripts on Monday , Tuesday, Wedensday, Thursday, Friday Only
:if (($result = "mon") || ($result = "tue") || ($result = "wed") || ($result = "thu") || ($result = "fri")) do={
:log warning "$result Detected , taking action that is disabling WAN links ... Powered by Syed.Jahanzaib"

## ACTION START to disable Wan links ##
# Start action
:local date;
:local time;
:local sub1 ([/system identity get name])
:local sub2 ([/system clock get time])
:local sub3 ([/system clock get date])
# GMAIL SMTP DYNAMIC Config Section, Make sure to change these values to match your's / Jz
# Gmail SMTP Address
:global GMAILSMTP
:set GMAILSMTP "64.233.167.108";
:local adminmail1 ADMIN1@hotmail.com
:local gmailid YOUR_GMAIL_ID@gmail.com
:local gmailuser YOUR_GMAIL_ID
:local gmailpass GMAIL_PASSWORD
:local gmailport 587
# Set EMAIL Subject Body
:local EMAILSUBJECT "$[/system clock get date] $[/system clock get time] -- ALL $ISP Links have been disabled at 8am."
:local EMAILBODY "ALL $ISP Links have been disabled at 8am.\n\n\n\n Auto Script for Mikrotik Designed by \n *** Syed_Jahanzaib ****"
# Set SMS body withou spaces, use plus sign for space
:local SMS "All+NAYATEL+Links+have+been+disabled+by+auto+script+at+8+am"
# Setting KANNEL INFO and SMS Number
:local KID "kannel"
:local KPASS "KANNEL_PASS"
:local KHOST "10.0.0.1:13013"
:local CELL1 "03333021909"
# Disable Wan Link(s)
/interface ethernet set ether1 disabled=yes
/interface set pppoe-nt1 disabled=yes
# Log action
:log error "$[/system clock get date] $[/system clock get time] -- ALL $ISP Links have been disabled at 8am by Auto Script."
# "Emailing the DOWN status. . . "
/tool e-mail send to="$adminmail1" password=$gmailpass subject="$EMAILSUBJECT" from=$gmailid server=$GMAILSMTP start-tls=yes body="$EMAILBODY"
# SENDING SMS for action DOWN
/tool fetch url="http://$KHOST/cgi-bin/sendsms\?username=$KID&password=$KPASS&to=$CELL1&text=$SMS"
## Disable ACTION ENDS HERE ##
##
# BUT If its Saturday & Sunday then do nothing
# else goto sleep
} else={
:log warning "Saturday - Sunday Detected , no need to disable wan links , Goto Sleep now ... Powered by Syed.Jahanzaib"
}
}
}

# END
# Syed Jahanzaib / UNICOM - 27-JAN-2018

FREERADIUS WITH MIKROTIK – Part #11 – Prepaid Hourly Accounts along-with the mighty COA !

$
0
0

one2bhour2brule

FREERADIUS WITH MIKROTIK – Part #1 – General Tip’s Click here to read more on FR tutorials …

Disclaimer: This is not a copy pasted post.  It’s purely personnel R&D work. I cannot guarantee if it will work at your end or not, but still you can have some fair idea ! I also assume you must have some fair knowledge on how freeradius & its modules works all together.!


Task:

Allow Prepaid hourly accounts as per following requirements …

#1

We want to allow `One hour access` to users, like coupons etc. and the Counting time should start after first login. Account will expire either 1 hour of its consumption OR if the user login first time but logouts before one hour limit reaches & date changes , then the account should expire !

Example we can provide a promo offer in a shopping mall that every customer who purchase 1000 Rs worth of goods, he can avail one hour of internet. If he consume 1 hour he should get disconnected, or if he consumes just 30 minutes, and left, the account will expire right after date changes to avoid re login next day.

#2

Once the limit is over, the FR should send disconnect message to Mikrotik so that user should disconnect on the FLY. weeeeeeeeeeeeeeee ….. 😉


~~ Hour account (for a single day) ~~

!

1# User Entry in radcheck table

First let add one Max-All-Session entry in the FR table.

mysql> select * from radcheck;
+----+----------+----------------------+----+----------------------+
| id | username | attribute | op | value |
+----+----------+----------------------+----+----------------------+
| 1 | zaib | Cleartext-Password | := | zaib |
| 2 | zaib | Max-All-Session | := | 10 |
| 2 | zaib | Access-Period     | := | 20 |
+----+----------+----------------------+----+----------------------+
2 rows in set (0.00 sec)

NOTE: In above , we are using 10 seconds max allowed time for a user , and if the user logout before time reaches, BUT the date changes , account will still expire because 20 seconds limit in access-period! It’s an example only ! Adjust according to your need. in Lab we have to perform fast in order to save time !


2# Create Module for Max-All-Session & Access-Period

We will create a module for session time  that will actually check user total used time from the radacct time.

[Note: We can also modify to check users table where we can make our custom column and let it update by radacct TRIGGER.]

nano /etc/freeradius/sites-enabled/default

& paste following

sqlcounter timelimit {
counter-name = Max-All-Session-Time
check-name = Max-All-Session
sqlmod-inst = sql
key = User-Name
reset = never
query = "SELECT SUM(AcctSessionTime) FROM radacct where UserName='%{%k}'"
}

Now create another module named accessperiod

& paste following (to check account expiry after first login, its in seconds)

sqlcounter accessperiod {
counter-name = Max-Access-Period-Time
check-name = Access-Period
sqlmod-inst = sql
key = User-Name
reset = never
query = "SELECT UNIX_TIMESTAMP() - UNIX_TIMESTAMP(AcctStartTime) FROM radacct WHERE UserName = '%{%k}' LIMIT 1"
}

Save & Exit.

3# Add above modules and SQL XLT entry in DEFAULT file

Edit DEFAULT file

nano /etc/freeradius/sites-enabled/default

& under authorize section , add the module name we made above.

timelimit
acessperiod

Now Move on to ACCOUNTING part and use following (its a SQL XLT commands to check if user have uptime_limit defined in the USERS table, if YES then simply compare the users usage from the radacct table with the uptime_limit defined in the users table.

If the condition meets, it should send POD to the NAS. This section is required only if you want to disconnect the user on the FLY.

accounting {
detail
unix
sql
exec
# Our Check entry Starts here -  CHECK Time limit from USERS table - ZaiB
# CHECK OVER TIME USER - zaiB
update control {
# Check if user have UPTIME limit defined or if its NULL
Tmp-Integer-0 := "%{sql:SELECT uptime_limit FROM users WHERE username='%{User-Name}'}"
# Get total time used from RADACCT table for this particular user
Tmp-Integer-1 := "%{sql:SELECT acctsessiontime AS Total FROM radacct where acctstarttime >= CURDATE() AND radacct.username='%{User-Name}'}"
Tmp-String-2 := "%{sql: SELECT value FROM users WHERE attribute='uptime_limit' AND username='%{User-Name}'}"
}
# If uptime limit is NULL means not deifned, then donot proceed further, Just ignore and move forward
if ("%{control:Tmp-Integer-0}" > 0){
# If uptime limit is defined in users table, then compare uptime with limit
if ("%{control:Tmp-Integer-1}" > "%{control:Tmp-String-2}"){
# If limit exceeds, send Disconnection packet, Simple is this, 🙂 zaib
update disconnect {
Framed-IP-Address = "%{Framed-IP-Address}"
User-Name = "%{request:User-Name}"
}
}
}
}

4# COA Section, to Disconnect on the FLY

First add COA section in the clients.conf so that we can provide FR the authority to send POD packet of disconnection to our beloved Mikrotik Router.

nano /etc/freeradius/clinets.conf

Paste following at the end of the file. Make sure to change the IP / Secret & port.

# COA Section / zaib , what a headache it was to enable COA, I am Quite a DUFFER for Sure : D
home_server example-coa {
type = coa
# Mikrotik IP Address
ipaddr = 101.11.11.253
# Radius INCOMING PORT, pay attention to this to make sure it matches with MT port
port = 1700
# Make sure that secret matches with MT as well
secret = testing123
# Now some Junk that I myself doesn't understand as well, but I just know that adding below is required to make COA works 😀
coa {
irt = 2
mrt = 16
mrc = 5
mrd = 30
}
}

 


5# Final Test

Now connect your windows workstation with the defined userid / password.

For test purposes I set accounting interim update to 10 seconds so that testing can be done quickly. Make a note of it ….

Now as soon as Mikrotik sends accounting request to FR, FR will do the matching case (that we defined) & if found condition to be TRUE, it will simply send DISCONNECT packet to the NAS.

Mikrotik LOG

mikrotik pod.JPG

RADIUS DEBUG

++update control {
sql_xlat
expand: %{User-Name} -> zaib
sql_set_user escaped user --> 'zaib'
expand: SELECT acctsessiontime AS Total FROM radacct where acctstarttime >= CURDATE() AND radacct.username='%{User-Name}' -> SELECT acctsessiontime AS Total FROM radacct where acctstarttime >= CURDATE() AND radacct.username='zaib'
rlm_sql (sql): Reserving sql socket id: 20
sql_xlat finished

rlm_sql (sql): Released sql socket id: 20
expand: %{sql:SELECT acctsessiontime AS Total FROM radacct where acctstarttime >= CURDATE() AND radacct.username='%{User-Name}'} -> 10
sql_xlat
expand: %{User-Name} -> zaib
sql_set_user escaped user --> 'zaib'
expand: SELECT value FROM users WHERE attribute='uptime_limit' AND username='%{User-Name}' -> SELECT value FROM users WHERE attribute='uptime_limit' AND username='zaib'

rlm_sql (sql): Reserving sql socket id: 19
rlm_sql_mysql: MYSQL check_error: 1054 received

rlm_sql (sql): database query error, SELECT value FROM users WHERE attribute='uptime_limit' AND username='zaib': Unknown column 'value' in 'field list'

rlm_sql (sql): Released sql socket id: 19
expand: %{sql: SELECT value FROM users WHERE attribute='uptime_limit' AND username='%{User-Name}'} ->

++} # update control = noop

++? if ("%{control:Tmp-Integer-0}" > "%{control:Tmp-String-1}")
expand: %{control:Tmp-Integer-0} -> 10
expand: %{control:Tmp-String-1} ->
? Evaluating ("%{control:Tmp-Integer-0}" > "%{control:Tmp-String-1}") -> TRUE

++? if ("%{control:Tmp-Integer-0}" > "%{control:Tmp-String-1}") -> TRUE
++if ("%{control:Tmp-Integer-0}" > "%{control:Tmp-String-1}") {
+++update disconnect {
expand: %{Framed-IP-Address} -> 1.1.1.255
expand: %{request:User-Name} -> zaib
+++} # update disconnect = noop
++} # if ("%{control:Tmp-Integer-0}" > "%{control:Tmp-String-1}") = noop
+} # group accounting = ok
Sending Accounting-Response of id 219 to 101.11.11.253 port 35756
WARNING: Empty pre-proxy section. Using default return values.
Sending Disconnect-Request of id 152 to 101.11.11.253 port 3799
Framed-IP-Address = 1.1.1.255
User-Name = "zaib"
Finished request 3.
Cleaning up request 3 ID 219 with timestamp +29
Going to the next request
rad_recv: Disconnect-ACK packet from host 101.11.11.253 port 3799, id=135, length=31
NAS-Identifier = "Win"
NAS-IP-Address = 101.11.11.253
# Executing section post-proxy from file /etc/freeradius/sites-enabled/default
+group post-proxy {
[eap] No pre-existing handler found
++[eap] = noop
+} # group post-proxy = noop
Finished request 2.
Going to the next request
Waking up in 2.3 seconds.
rad_recv: Disconnect-NAK packet from host 101.11.11.253 port 3799, id=152, length=37
Error-Cause = Session-Context-Not-Found
NAS-Identifier = "Win"
NAS-IP-Address = 101.11.11.253
# Executing section post-proxy from file /etc/freeradius/sites-enabled/default
+group post-proxy {
[eap] No pre-existing handler found
++[eap] = noop
+} # group post-proxy = noop
Finished request 3.
Going to the next request
Ready to process requests.

& on the next LOGIN attempt, we will observe following …

Reply-Message = "Your maximum never usage time has been reached"

off-course you can customized this message as well….


Regard’s
Syed Jahanzaib

Access other OP portal via Mikrotik Load Balancer

$
0
0

portal routing in pcc.jpg

Disclaimer: I receive many emails from local operators on howto access other operators media sharing portal so that there local users can access them. Rather then replying each one separately & Due to time shortage, I am posting a simple method on how you can let your users access the outer operator media portal via your load balance mikrotik.  It is highly recommended to first search for the target web site/servers ip either using trace-route or wire shark. you have to conduct lengthy tracing by simply first try to connect with the target web site and start downloading multiple files, now using either TORCH, or using WIRE SHARK, you can get ips of all the servers which is being accessed by the torrent/idm which is connecting wit the target services. just make a note of these ip’s and add them in list either ip by ip or by /24 subnet.

Also It is recommended to use Mark Connections / Packets and Mark Routing. This way you wont have to create addition rules. So following pseudo codes is highly recommended to fit yourself in a famous quote that says `Work Smarter , not harder …`

Regard’s
Syed Jahanzaib

Example of WIRESHARK is posted bellow ….

wire-shark-example


Scenario:

We are using Mikrotik as pppoe server and dual vdsl links as WAN load balancer. We have acquired another Operator X line just to access there Entertainment portal which is great in media sharing files including video games etc. We want to let our user access there portal using our mikrotik without interfering with any other network.


Solution:

Quick & Dirty method. You should refine it when deploying in production environment.

We have configured an simple wifi router along with Operator X service in it. Now connect this router LAN line in your mikrotik (example Port 12).

IP Scheme:

  • Mikrotik LAN IP for pppoe users : eth0 > 192.168.0.1
  • Mikrotik PPPoE IP series (allowed users for internet) : 192.168.200.0/24
  • Mikrotik WAN-1 IP Series for DSL1 : eth1 > 192.168.1.1
  • Mikrotik WAN-2 IP Series for DSL2 : eth2 > 192.168.2.1
  • Mikrotik eth12 IP Series for Portal X : eth12 > 192.168.12.2 ( > 192.168.12.1 is wifi router with local OP service)
  • PORTAL-X IP Series: Web Portal – 123.123.123.0/24  Download servers – 172.17.1.0/24

Note:
For DNS, you can either use the OP-X dns servers (or wifi router as your dns as wifi router will get the DNS dynamically from the OP-X, or you can make static dns entries in your mikrotik dns server , and make sure all of your clients are using your mikrotik as there preferred dns server, you can also create a forced router to redirect all outgoing dns requests to your mirkotik. whatever is easier for you 🙂


Code!

# PPPoE Users IP List to access internet/portal
/ip firewall address-list
add address=192.168.200.0/24 comment="Allowed Users to Use Internet" list="allowed users"
# Add Portal X IP Series, you can get there list by inspecting torrent files, or using WIRESHARK
add address=123.123.123.0/24 list=portalx_list
add address=172.17.1.0/24 list=portalx_list

# Accept the PORTAL X packets to avoid processing them in PCC, then using routes we can route them via wifi router
/ip firewall mangle
add action=accept chain=prerouting comment="ACCEPT portalx_list PACKETS FROM PROCESSING THEM IN PCC - ZAIB" dst-address-list=portalx_list src-address-list="pppoe_allowed_users"

# Allow requests going to Portal X interface (to wifi router with OP X service)
/ip firewall nat
add action=masquerade chain=srcnat comment=ALLOW_ACCESS_TO_portalx_list_INTERFACE out-interface=eth12 src-address-list="pppoe_allowed_users

# Since we have excluded the Portal.X from PCC, therefor we have to create ROUTE for these packets
# So that these packets should route via Wi.Fi Router (connected with OP X service)
/ip route
add distance=1 dst-address=123.123.123.0/24 gateway=192.168.12.1 comment=route_for_portalx_site_going_via_local_wifi_router
add distance=1 dst-address=172.17.1.0/24 gateway=192.168.12.1 comment=route_for_portalx_site_going_via_local_wifi_router

# In the end , simply create QUEUE to allow more bandwidth going to Portal X servers,
# Again, if you had used packet marking, then you can use marked pkts & use single queue, more efficient
/queue simple
add max-limit=1G/1G name="portalx_list.torrent queue-1G" target=172.17.17.0/24
add max-limit=1G/1G name=portalx_list-public-ips-1G target=123.123.123.0/24

Advertisements

FREERADIUS WITH MIKROTIK – Part #12 – Happy Hours !

$
0
0

fre2

happy hours

FREERADIUS WITH MIKROTIK – Part #1 – General Tip’s Click here to read more on FR tutorials …


* SCENARIO:

We have a full working freeradius based billing system which have multiple Quota base services assigned to users. All users accounting is stored in radacct table as usual, but we have a separate column named qt_used in users table which is updates via radacct trigger when accounting updates arrives from the NAS. This column is checked by an external script every 10 minutes & if it finds qt_used value above then qt_total then it simply change user group to EXPIRED group & disconnects users from NAS by sending COA / POD.

* TASK:

Our bandwidth remains empty in night therefore we want to provide some extra benefits to quota base users by BYPASSING quota limit & counting in specific late night hours.

Example: There should be no QUOTA counting for all quota base users from 00:00 (midnight) till 8am (morning).

* Possible SOLUTION !

Like always, this may not be best optimal solution to achieve the task, but still it may work. Be sure took for other better approach rather then this sol.

Following is our USERS table

mysql> describe users;
+-------------------+--------------+------+-----+-------------------+-----------------------------+
| Field | Type | Null | Key | Default | Extra |
+-------------------+--------------+------+-----+-------------------+-----------------------------+
| id | int(10) | NO | PRI | NULL | auto_increment |
| username | varchar(128) | NO | UNI | NULL | |
| password | varchar(32) | NO | | NULL | |
| firstname | text | NO | | NULL | |
| lastname | text | NO | | NULL | |
| email | text | NO | | NULL | |
| mobile | text | NO | | NULL | |
| cnic | text | NO | | NULL | |
| srvname | text | NO | | NULL | |
| srvid | int(3) | NO | | NULL | |
| expiration | date | YES | | NULL | |
| mac | varchar(30) | NO | | NULL | |
| bwpkg | varchar(256) | NO | | NULL | |
| is_enabled | int(1) | NO | | NULL | |
| is_days_expired | int(1) | NO | | NULL | |
| is_qt_expired | int(1) | NO | | NULL | |
| is_uptime_expired | int(1) | NO | | NULL | |
| qt_used | varchar(20) | NO | | NULL | |
| uptime_used | varchar(32) | NO | | NULL | |
| owner | text | NO | | NULL | |
| vlanid | varchar(32) | NO | | NULL | |
| createdon | timestamp | NO | | CURRENT_TIMESTAMP | on update CURRENT_TIMESTAMP |
+-------------------+--------------+------+-----+-------------------+-----------------------------+
22 rows in set (0.00 sec)

& following is our services table

mysql> describe services;
+--------------------+--------------+------+-----+-------------------+-----------------------------+
| Field | Type | Null | Key | Default | Extra |
+--------------------+--------------+------+-----+-------------------+-----------------------------+
| srvid | int(10) | NO | PRI | NULL | auto_increment |
| srvname | varchar(128) | NO | MUL | NULL | |
| descr | varchar(128) | YES | | other | |
| enabled | varchar(1) | NO | | NULL | |
| expdays | varchar(3) | NO | | NULL | |
| dlimit | varchar(30) | NO | | NULL | |
| ulimit | varchar(30) | NO | | NULL | |
| qt_enabled | tinyint(1) | NO | | NULL | |
| tot_qt | int(128) | NO | | NULL | |
| free_quota_enabled | tinyint(4) | NO | | NULL | |
| free_qt_start_time | time | NO | | NULL | |
| free_qt_end_time | time | NO | | NULL | |
| ippool | varchar(30) | NO | | NULL | |
| createdon | datetime | NO | | CURRENT_TIMESTAMP | on update CURRENT_TIMESTAMP |
+--------------------+--------------+------+-----+-------------------+-----------------------------+
14 rows in set (0.00 sec)

* MYSQL TRIGGER on RADACCT table for AFTER UPDATE ACTION. [ This is our magical code 😛 ]

Ok, now we will create a TRIGGER on radacct table for AFTER UPDATE action.
[apply it in mysql shell on radius table]

BEGIN
SET @ctime = (select current_time());
SET @srvid = (select srvid from users where username =New.Username);
SET @st = (select free_qt_start_time from services where srvid=@srvid);
SET @et = (select free_qt_end_time from services where srvid=@srvid);
if (@ctime > @st)
OR (@ctime < @et)
then
UPDATE users set qt_used = qt_used + NEW.acctoutputoctets WHERE username = New.username;
# below section ELSE is not required, I just added it for ON THE FLY troubleshooting]
else
insert into log (data,msg) values ('zaib','happy_hours_time_not_matched');
END IF;
END
# Syed Jahanzaib / aacable at hotmail dot com

Current time is 11:00{am}, inspect the mysql.log & you will find following (quota will not update because time doesnt matches)

2018-02-19T06:18:26.797015Z 51 Query UPDATE radacct SET framedipaddress = '192.168.50.255', acctsessiontime = '1841', acctinputoctets = '0' > 32 | '122655', acctoutputoctets = '0' > 32 | '145069' WHERE acctsessionid = '8100000e' AND username = 'zaib' AND nasipaddress = '101.11.11.253'
2018-02-19T16:18:26.797819Z 51 Query SET @ctime = (select current_time())
2018-02-19T16:18:26.797889Z 51 Query SET @srvid = (select srvid from users where username =New.Username)
2018-02-19T16:18:26.798009Z 51 Query SET @st = (select free_qt_start_time from services where srvid=@srvid)
2018-02-19T16:18:26.798092Z 51 Query SET @et = (select free_qt_end_time from services where srvid=@srvid)
2018-02-19T16:18:26.798165Z 51 Query insert into log (data,msg) values ('zaib','happy_hours_time_not_matched)

.

& if the radacct updates came in between happy hours then we will observe following (Quota will update accordingly)

2018-02-19T06:19:31.818111Z 98 Query UPDATE radacct SET framedipaddress = '192.168.50.255', acctsessiontime = '1906', acctinputoctets = '0' << 32 | '130274', acctoutputoctets = '0' << 32 | '161901' WHERE acctsessionid = '8100000e' AND username = 'zaib' AND nasipaddress = '101.11.11.253'
2018-02-19T06:19:31.818500Z 98 Query SET @ctime = (select current_time())
2018-02-19T06:19:31.818574Z 98 Query SET @srvid = (select srvid from users where username =New.Username)
2018-02-19T06:19:31.818680Z 98 Query SET @st = (select free_qt_start_time from services where srvid=@srvid)
2018-02-19T06:19:31.818758Z 98 Query SET @et = (select free_qt_end_time from services where srvid=@srvid)
2018-02-19T06:19:31.818833Z 98 Query UPDATE users set qt_used = qt_used + NEW.acctoutputoctets WHERE username = New.username

 


Regard's
Syed Jahanzaib ~

.


High CPU load when PPPoE sessions disconnects in Mikrotik

$
0
0

stress


Disclaimer:

Every Network is different , so one solution cannot be applied to all. Therefore try to understand logic & create your own solution as per your network scenario. Just dont follow copy paste.

If anybody here thinks I am an expert on this stuff, I am NOT certified in anything Mikrotik/Cisco/Linux or Windows. However I have worked with some core networks and I read & research & try stuff all of the time. So I am not speaking/posting about stuff I am formerly trained in, I pretty much go with experience and what I have learned on my own. And , If I don’t know something then I read & learn all about it.

So , please don’t hold me/my-postings to be always 100 percent correct. I make mistakes just like everybody else. However – I do my best, learn from my mistakes and always try to help others

 


Scenario-1:

We are using Mikrotik CCR as PPPOE/NAS. We are using public ip routing setup so each user is assigned public ip via pppoe profile.

Scenario-2:

We are using single Mikrotik CCR as PPPOE/NAS. We have local dsl service therefore NATTING is also done on the same router.


Problem:

When we have network outages like light failure in any particular area , in LOG we see many PPPoE sessions disconnects with ‘peer not responding‘ messages. Exactly at this moments, our NAS CPU usage reaches to almost 100% , which results in router stops passing any kind of traffic. This can continue for a minute or so on.

As showed in the image below …

pppoe high cpu usage

If you are using Masquarade /NAT on the router, that is the problem. When using Masquarade, RouterOS has to do full connection tracking recalculation on EACH interface connect/disconnect.

So if you have lots of PPP session connecting/disconnecting, connection tracking will constantly be recalculated which can cause high CPU usage. When interfaces connect/disconnect, in combination with NAT, it gives you high CPU usage.


Solution OR Possible Workarounds :

  • If you have private ip users with natting, Stop using Masquarade on same router that have a lot of dynamic interfaces. Just DO NOT use NAT on any router that have high number of connecting/disconnecting interfaces. Place an additional router connected with your PPPoE NAS, and route NAT there.
    Example: Add another router & perform all natting on that router by sending marked traffic from private ip series to that nat router. Setup routing between the PPPoE NAS and the NAT router.
  • If all of your clients are on public IP , you can simply Turn Off connection tracking completely. This is the simplest approach. You can exempt your specific public pool from connection tracking as well.

ct

 

  • Any device that is CORE device or Gateway on your network, It should be assigned to perform one job only. Try not to mix multiple functions in one device. This will save you from later headache of troubleshooting.

Please read this …

Features affected by connection tracking

  • NAT
  • firewall:
    • connection-bytes
    • connection-mark
    • connection-type
    • connection-state
    • connection-limit
    • connection-rate
    • layer7-protocol
    • p2p
    • new-connection-mark
    • tarpit
  • p2p matching in simple queues

So if you will turn OFF the connection tracking, above features will stop working.


– Code Snippet:

Some working example of excluding your public pool from connection tracking

  • First make sure Connection Tracking is set to AUTO
/ip firewall connection tracking set enabled=auto

 

  • Then make a address list which should have your users ip pool so that we can use this list as an Object in multiple rules later.
/ip firewall address-list
add address=1.1.1.0/24 list=public_pool
#add address=2.1.1.0/24 list=public_pool

 

  • Now create rule to turn off connection tracking from our public ip users witht the RAW table
/ip firewall raw
add action=notrack chain=prerouting src-address-list=public_pool

That’s it!



Some Tips for General Router Management

  • Turn off all non essential services that are not actually being used or needed. Services place an additional CPU load on any system. Example, you can move your DHCP role to cisco switches for better response , also for intervlan routing it is highly recommended., Also if your ROS is acting as DNS as well, then move DNS role to dedicated dns server like BIND etc. This will free up some resources from the core system
  • Use 10-gig network cards instead of 1-gig / Use 1-gig network cards instead of 100 meg
    / Use 100 meg network cards instead of 10 meg
  • Disable STP if it is not needed. Now this is highly debatable part I know 🙂

Regard's
Syed Jahanzaib ~

FREERADIUS WITH MIKROTIK – Part #13 – Detecting user device vendor based on MAC address

$
0
0

fre2

mac-address

1- identify vendor from mac

 

FREERADIUS WITH MIKROTIK – Part #1 – General Tip’s Click here to read more on FR tutorials …

Disclaimer! This is important!

Every Network is different , so one solution cannot be applied to all. Therefore try to understand logic & create your own solution as per your network scenario. Just dont follow copy paste.

If anybody here thinks I am an expert on this stuff, I am NOT certified in anything Mikrotik/Cisco/Linux or Windows. However I have worked with some core networks and I read , research & try stuff all of the time. So I am not speaking/posting about stuff I am formerly trained in, I pretty much go with experience and what I have learned on my own. And , If I don’t know something then I read & learn all about it.

So , please don’t hold me/my-postings to be always 100 percent correct. I make mistakes just like everybody else. However – I do my best, learn from my mistakes and always try to help others

Regard's
Syed Jahanzaib~

* Scenario:

We have a generic freeradius based billing system in place. in RADACCT table, we have a trigger that fire-up after successfull connection made from user device & it inserts user device MAC address in radcheck table for MAC validation & restriction.

* Requirement:

For better management , control & reporting purposes we want to add more checks and controls by adding user calling device VENDOR name in an additional column so that we can have idea what devices are most common in our network, or to detect any device that is prohibited by the company SOP or policy example commercial grade routers to minimize misuse of our residential services. Based on this reporting  we can prohibit connecting these devices as well if required.

To fulfill this we will do following

  1. Create a TRIGGER on RADACCT that will executes after INSERT record (like when user will connect to system successfully)
  2. Create DB and upload vendors mac address data so that we can query for vendor name locally (alternate approach is to simple use any Perl or bash script to look up vendor name dynamically on the fly form the internet)

 


OK before creating TRIGGER we must learn or understand our USER table where mac related will be added or update. One example for such table is showed below …

root@radius:/temp# mysql -uroot -pSQLPASS -e "use radius; describe users;"

+-------------------+--------------+------+-----+-------------------+-----------------------------+
| Field | Type | Null | Key | Default | Extra |
+-------------------+--------------+------+-----+-------------------+-----------------------------+
| id | int(10) | NO | PRI | NULL | auto_increment |
| username | varchar(128) | NO | UNI | NULL | |
| password | varchar(32) | NO | | NULL | |
| firstname | text | NO | | NULL | |
| lastname | text | NO | | NULL | |
| email | text | NO | | NULL | |
| mobile | text | NO | | NULL | |
| cnic | text | NO | | NULL | |
| srvname | text | NO | | NULL | |
| srvid | int(3) | NO | | NULL | |
| expiration | date | YES | | NULL | |
| mac | varchar(30) | NO | | NULL | |
| macvendor | varchar(128) | NO | | NULL | |
| bwpkg | varchar(256) | NO | | NULL | |
| pool | varchar(128) | YES | | other | |
| is_enabled | int(1) | NO | | NULL | |
| is_days_expired | int(1) | NO | | NULL | |
| is_qt_expired | int(1) | NO | | NULL | |
| is_uptime_expired | int(1) | NO | | NULL | |
| qt_total | varchar(32) | NO | | NULL | |
| qt_used | varchar(20) | NO | | NULL | |
| uptime_limit | varchar(20) | NO | | NULL | |
| uptime_used | varchar(32) | NO | | NULL | |
| owner | text | NO | | NULL | |
| vlanid | varchar(32) | NO | | NULL | |
| createdon | timestamp | NO | | CURRENT_TIMESTAMP | on update CURRENT_TIMESTAMP |
+-------------------+--------------+------+-----+-------------------+-----------------------------+

In this post we have interest in two columns named mac and macvendor


Now we will create TRIGGER with our so called magical code 😉

1- TRIGGER for radacct table

CREATE TRIGGER `chk_mac_after_insert` AFTER INSERT ON `radacct`
 FOR EACH ROW BEGIN
# Check if user mac is already added in radcheck table,
SET @mac = (SELECT count(*) from radcheck where username=New.username and attribute='Calling-Station-ID');
# If there is no entry for mac, then update mac in radcheck table, so that in future only this mac will be able to connect with that username
IF (@mac = 0) THEN 
INSERT into radcheck (username,attribute,op,value) values (NEW.username,'Calling-Station-ID',':=',NEW.callingstationid); 
# add mac in users table for general record purpose
UPDATE users SET mac = NEW.callingstationid where username = NEW.username;
# trim mac for first 3 strings to detect vendor company
SET @mactrim = (select LEFT(mac, 8) from users where username=New.username);
# get vendor name from mac db table
SET @macvendor1 = (select vendor from macdb where oui=@mactrim);
# Update vendor name in user table
UPDATE users SET macvendor=@macvendor1 where username = NEW.username;
END IF;
END
#SYED Jahanzaib - 
2- triger

2- Create MAC Address Database for VENDOR Lookup

This is a bit debatable part, I prefer to have local database for vendor mac addresses so that all lookup should be done locally rather then using any API for mac lookup. But if you want to prefer using internet base lookup , then you can use perl, bash or any other method to do lookup one example for internet lookup is as follows …

curl http://api.macvendors.com/70-54-D2-16-A5-D9
#OUTPUT

PEGATRON CORPORATION

OK lets move forward to create local DB for mac and vendor lookup.

  • Install required modules
apt-get install python-mysqldb python-minimal
  • Download the MAC address OUI from the IEEE site.
cd /tmp
wget http://standards.ieee.org/develop/regauth/oui/oui.txt
  • Once download is done, Create script that will insert mac records into separate table named ​mactable
touch mactosql.sh
chmod +x touch mactosql.sh

Now edit file

nano mactosql.sh

Paste following

# 09-07-2016
# ouiUpdate, grabs the iee oui list and adds it to a table in mysql database.
# Provides no syntax checking of the input file, use with caution
# based on load_iou_to_pgsql.py https://gist.github.com/pwldp/dee6d5f9868a1dd1d076
# Usage:
# python ouiUpdateMysql.py

import MySQLdb
import re
import urllib
import sys

# Globals
# Database Handle, again, no need to change them; Only Declarations
DB = ''
DB_CURSOR = ''
OUI_URL = "http://standards.ieee.org/develop/regauth/oui/oui.txt"
OUI_FILE = "oui.txt"

# Database Related Globals
DB_NAME='radius'
DB_USERNAME='root'
DB_PASSWORD='SQL_PASSWORD'
DB_TABLE='macdb'
DB_HOST='localhost'

def main():
global DB
global DB_CURSOR
global DB_TABLE
# Download oui.txt Comment out the two lines below if you do not want to download the file and use a offline copy
#####
# print "Downloading ",OUI_URL
# urllib.urlretrieve(OUI_URL, OUI_FILE)
#####
##connect to db
try:
## Opening connection to Database with pre-configured Database, Host, User and Password
DB = MySQLdb.connect(host=DB_HOST, user=DB_USERNAME, passwd=DB_PASSWORD, db=DB_NAME)
DB_CURSOR = DB.cursor()
except:
sys.exit('Unable to Open Connection to Database')

# Drop table and start from new
DB_CURSOR.execute("DROP TABLE IF EXISTS ouiList")
# Create table as per requirement
sql ="""CREATE TABLE IF NOT EXISTS ouiList (
id int(6) NOT NULL AUTO_INCREMENT,
oui varchar(8) DEFAULT NULL,
vendor varchar(100) DEFAULT NULL,
PRIMARY KEY (id)
) ENGINE=InnoDB"""

# make the new table
DB_CURSOR.execute(sql)

# parsing oui.txt data and adding to table
print "Parsing data..."
with open(OUI_FILE) as infile:
print "Inserting data into table, please wait..."
for line in infile:
#do_something_with(line)
if re.search("(hex)", line):
try:
mac,vendor = line.strip().split("(hex)")
except:
mac = vendor = ''
#print line.strip().split("(hex)")
#print mac.strip().replace("-",":").lower(), vendor.strip()
if mac!='' and vendor!='':
sql = "INSERT INTO ouiList "
sql+= "(oui,vendor) "
sql+= "VALUES ("
sql+= "'%s'," % mac.strip().replace("-",":").lower()
sql+= "'%s'" % vendor.strip().replace("'","`")
sql+= ")"
#print sql
try:
DB_CURSOR.execute(sql)
DB.commit()
except Exception, e:
DB.rollback()
print "Not inserted because error: "

# count the number of lines in the table
sql = ("SELECT count(id) FROM ouiList WHERE 1")
DB_CURSOR.execute(sql)
result = DB_CURSOR.fetchone()
if result:
print 'Completed: Total lines in table is: {0}'.format(result[0])
else:
print "Unable to count lines in table"

infile.close()
DB_CURSOR.close()
DB.close()
#

if __name__=="__main__":
main()

SAVE & EXIT. Now execute it.

python mactosql.sh

If no error occurs you will be seeing a new table named macdb with all vendors names as well 🙂

db import successfull.JPG

 

to get trimmed OUI, use following

sourcecode]mysql -uroot -pSQLPASS –skip-column-names -s -e “use radius; select LEFT(mac, 8) from users;”/sourcecode]


Now connect any user as normal, and see the mysql.log file

2018-02-20T06:41:04.593739Z 24 Query INSERT INTO radacct (acctsessionid, acctuniqueid, username, realm, nasipaddress, nasportid, nasporttype, acctstarttime, acctstoptime, acctsessiontime, acctauthentic, connectinfo_start, connectinfo_stop, acctinputoctets, acctoutputoctets, calledstationid, callingstationid, acctterminatecause, servicetype, framedprotocol, framedipaddress, acctstartdelay, acctstopdelay, xascendsessionsvrkey) VALUES ('81100003', '702d22ac0a080f57', 'zaib', '', '101.11.11.253', '9', 'Ethernet', '2018-02-20 11:41:04', NULL, '0', 'RADIUS', '', '', '0', '0', 'service1', '00:0C:29:B9:D8:A0', '', 'Framed-User', 'PPP', '192.168.50.255', '0', '0', '')

# As soon we receive entry for INSERT, TRIGGER will fire-up, see below log

2018-02-20T06:41:04.594676Z 24 Query SET @mac = (SELECT count(*) from radcheck where username=New.username and attribute='Calling-Station-ID')

2018-02-20T06:41:04.594871Z 24 Query INSERT into radcheck (username,attribute,op,value) values (NEW.username,'Calling-Station-ID',':=',NEW.callingstationid)
2018-02-20T06:41:04.595020Z 24 Query UPDATE users SET mac = NEW.callingstationid where username = NEW.username

2018-02-20T06:41:04.595151Z 24 Query SET @mactrim = (select LEFT(mac, 8) from users where username=New.username)
2018-02-20T06:41:04.595256Z 24 Query SET @macvendor1 = (select vendor from macdb where oui=@mactrim)

2018-02-20T06:41:04.607786Z 24 Query UPDATE users SET macvendor=@macvendor1 where username = NEW.username

& FINALLY we will see records in USERS table that will be displayed on FRONTEND 🙂

1- identify vendor from mac

Bursting with Mikrotik Burst ^o^

$
0
0

Disclaimer! This is important!

Every Network is different , so one solution cannot be applied to all. Therefore try to understand logic & create your own solution as per your network scenario. Just dont follow copy paste.

If anybody here thinks I am an expert on this stuff, I am NOT certified in anything Mikrotik/Cisco/Linux or Windows. However I have worked with some core networks and I read , research & try stuff all of the time. So I am not speaking/posting about stuff I am formerly trained in, I pretty much go with experience and what I have learned on my own. And , If I don’t know something then I read & learn all about it.

So , please don’t hold me/my-postings to be always 100 percent correct. I make mistakes just like everybody else. However – I do my best, learn from my mistakes and always try to help others

Regard's
Syed Jahanzaib~

Mikrotik Burst feature provides predefined extra bandwidth for a limited period of time IF the user remains under the burst threshold limit, or else He will be limited to his max–limit package.

It is best explained here

https://wiki.mikrotik.com/wiki/Manual:Queues_-_Burst

 

Real life Example:


Explanation !

  • User IP (1-TARGET) on which this queue will be implemented
  • When this IP will start downloading he can reach download rate of 512 kbps (3-BURST_LIMIT)
  • Until he continue to do so for a minute (5-BURST_TIME) (period of time, in seconds, over which the average data rate is calculated. (This is NOT the time of actual burst, so on avg it will become 30s)
  • That is on an average basis his download remains 256 kbps (4-BURST_THRESHOLD) for a minute, (average of 30 seconds)
  • Then he will be get back limited to his max-limit (2-MAX_LIMIT)
  • When a user doesn’t use the traffic at all and 30 second average goes to 0 so the next time traffic is requested then it will be at the Burst speed (3-BURST_LIMIT).

 

Small BURST TIME may not give you correct results. So use reasonable time. in this example I used shorter time for demonstration purposes. A large burst isn’t a problem technically, it’s more of a business decision.

To calculate burst and relates values, download this excel sheet named “MikroTik burst simulator.xlsx” from my google drive & try it yourself … you will get clear picture


Another Example by joshaven! so that you can better understand —-


Now look at its Demo ,

 

 

Advertisements

Renew DHCP lease if Gateway not responding

$
0
0

automation

Nothing Fancy , just short notes for script command reference purposes!

Following is a very simple 2 minutes instant cooked noddle type script. It will simply check the gateway status acquired by the dhcp-client using  ARP ping, , if it fails, then it will simply try to renew the ip by release/renew. Its very basic level, but its interesting to see how Mikrotik can really help you in day to day task by facilitated with the Tik level scripting !

Regard’s
Syed Jahanzaib

# Mikrotik Script - Tested with 5.x
# Script to check default gateway acquired by dhcp client on specific interface,
# Lot of room for improvements and modification but following was enough for some particular task,
# You can add BOUND status as well too, but i wanted this particular checking, you can add whatever you like
# Syed Jahanzaib == aacable AT hotmail DOT com - https:// aacable DOT wordpress DOT com
# Feb,2018
# Setting Variables

# Set Interface name which will get DHCP ip , This is the only option you may need to modify
:local INTERFACE "wan1"

# Number of Ping Count, how many times mikrotik should ping the target device
:local PINGCOUNT "5"

# Ping threshold, how many values should set alert, like if 5 out of 5 goes out
:local PINGTS "5"

:local i 0;
#:local i value=0;
:local F 0;
:local date;
:local time;

:log info "Checking default gateway for $INTERFACE interfaces."
:local DHCPGW [ /ip dhcp-client get [/ip dhcp-client find where interface=$INTERFACE ] gateway ]

# IF there is no default gateway on dhcpclient interface or if interface is disabled, then error
:if ([:len $DHCPGW] = 0) do={
:log error "No DEFAULT gateway found on $INTERFACE interface @ $date $time ..."
# Try to renew ip
/ip dhcp-client release $INTERFACE
/ip dhcp-client renew $INTERFACE
# Exit the script without further process ... I found this recently because in mikrotik there is no EXIT 1
error :error
}

# PING host $PINGCOUNT times
:for i from=1 to=$PINGCOUNT do={
:if ([/ping arp-ping=yes interface=$INTERFACE $DHCPGW count=1]=0) do={:set F ($F + 1)}
:delay 1;
};

# If no ping found then give error and do action
:if (($F=$PINGTS)) do={
:log error "PING to $DHCPGW via $INTERFACE is DOWN! @ $date $time "

# Take action if unable to ping gateway
/ip dhcp-client release $INTERFACE
/ip dhcp-client renew $INTERFACE
} else={
:log warning "PING to $DHCPGW via $INTERFACE is UP! @ $date $time "
}

Power of Open Source / Traffic Controlling with TC

$
0
0

speed meter

 

pwoerful pawn

Living in the Dark & Playing  & with the Open Source is Fun & Amazingly Powerful ! z@iB


Scenario:

We have few Linux base FTP/HTTP base media file sharing server setup. User can access these server by bypassing the router as we donot want to put load on the router itself & user should access these servers from switch level by intervlan routing setup.

Problem:

Since the FTP data is in many terabytes and hundreds of users are accessing it, this is creating bottleneck for server’s network/storage.

Solution:

On Linux we can use TRAFFIC CONTROLLER (TC) to limit bandwidth on per user IP basis.  We will create a simple bash script which will do the following in sequence …

  • FTP Server IP: 101.11.11.254
  • Clear any existing shaping rules
  • Create 4 bandwidth packages as following
    – 1024kbps  [for General Users subnet 101.11.11.0/24]
    – 2048kbps  [for Support Staff  subnet 101.11.50.0/24]
    – 512kbps     [for ICMP, so that ping should work fine , Protocol base]
    – 128kbps    [for any unknown IP series, that is no define above, for ALL 0.0.0.0/0.0.0.0]
  • Assign ip subnet to these packages (/24 and it will create per ip base rules via LOOP statement)
  • In this example, we are limiting Bandwidth in one way only, means packets sending from server to client will be applied limitation .. For both ways just duplicate rules and use src as well.

Create , the script!

mkdir /temp
touch /temp/shaper.sh
chmod +x /temp/shaper.sh

Now Edit

nano /temp/shaper.sh

& paste following,

#!/bin/bash
# Traffic Shaping Using `TC` in Linux / This was made for specific requirements.
# ITs customzied designed to fulfill some specific requirements & it went well.
# Following Script will limit outgoing bandwidth for specific ip range series.
# It will also allow ICMP more bandwidth so that ping to local server may not delay or imtout.
# It will then limit all other ips that are not listed to minimum bandwidth that is 128kbps.
# Adjust is as per your requirements.
# Syed Jahanzaib / aacable @ hotmail . com / https://aacable.wordpress.com :~)
# 23-Febraury-2018

# Enable following to see debug
# set -x

# Local Network Interface name
INT="ens33"
INT_SPEED="100Mbit"

# First IP Range & its package with class
IP1="101.11.11"
IP1_PKG="1mbit"
IP1_CLASSID="1:10"

# Second IP Range & its package with class
IP2="101.11.50"
IP2_PKG="2mbit"
IP2_CLASSID="1:20"

# Separate Bandwidth for ICMP protocol to minimze delay in PING/ICMP packets
ICMP_PKG="512kbit"
ICMP_CLASSID="1:200"

# Other Default Package Line, means all other unmarked IP's or traffic
UNKNOWN_PKG="128kbit"
UNKNOWN_CLASSID="1:201"

if [[ "$1" != "start" && "$1" != "stop" && "$1" != "status" ]]; then
echo "Usage:

./shaper.sh start
./shaper stop
./shaper status

by Jz!"
exit 1
fi

if [ $1 == "stop" ]; then
echo "Clearing all existing Queues on user request ...

Done!"
tc qdisc del dev $INT root 2> /dev/null > /dev/null
exit 1
fi

if [ $1 == "status" ]; then
echo "Showing Current TC Class and Filter Status
=========================================="
tc class show dev $INT
echo "

"
tc -s -d class show dev $INT
echo "=========================================="

exit 1
fi

if [ $1 == "start" ]; then
echo "Starting Shaper ...
Clearing all existing Queues for re-implementation ...
=========================================="
"

# clean existing down- and uplink qdiscs, hide errors
tc qdisc del dev $INT root 2> /dev/null > /dev/null

echo "
Adding Queues ...."
# Create egress shaping for all required classes
tc qdisc add dev $INT root handle 1: htb default 20 r2q 50
tc class add dev $INT parent 1: classid 1:1 htb rate $INT_SPEED ceil $INT_SPEED
tc class add dev $INT parent 1:1 classid $IP2_CLASSID htb rate $IP1_PKG ceil $IP1_PKG prio 100
tc class add dev $INT parent 1:1 classid $IP1_CLASSID htb rate $IP2_PKG ceil $IP2_PKG prio 100
tc class add dev $INT parent 1:1 classid $ICMP_CLASSID htb rate $ICMP_PKG ceil $ICMP_PKG prio 100
tc class add dev $INT parent 1:1 classid $UNKNOWN_CLASSID htb rate $UNKNOWN_PKG ceil $UNKNOWN_PKG prio 100

# ICMP reply Adjustment so ping may not timeout or high delay may not occur at client end for this server local IP
tc filter add dev $INT parent 1: protocol ip prio 100 u32 match ip protocol 1 0xff flowid $ICMP_CLASSID

# IP-1 series limitation applies here ...
for (( i = 1 ; i <= 255; i++ ))
do
b=$i
tc filter add dev $INT parent 1: protocol ip prio 100 u32 match ip dst $IP1.$i flowid $IP1_CLASSID
done

# IP-2 series limitation applies here ...
for (( i = 1 ; i <= 255; i++ ))
do
b=$i
tc filter add dev $INT parent 1: protocol ip prio 100 u32 match ip dst $IP2.$i flowid $IP2_CLASSID
done

# Send every unknown IP to classid UNKNOWN_CLASS that is lowest. so that any IP that is not listed above will get htis lowest bandwidth
tc filter add dev $INT parent 1: protocol ip prio 100 u32 match ip dst 0.0.0.0/0.0.0.0 flowid $UNKNOWN_CLASSID

echo "
Done!"
fi

Result:

linux TC down-result.jpg

 

Advertisements

Short Notes on LibreNMS !

$
0
0

libre dashboard

This post contains short notes & step by step guide on open source network omonitoring system called LIBRENMS , it is based on Php/MySQL/Snmp.


STEP BY STEP iNSTALLATION FOR LIBRENMS

OS: Ubuntu 16 (Server Edition/64bit)

First install required support packages

apt-get update
# Now install all the required components in a single GO ...
apt install -y apache2 composer fping git graphviz imagemagick libapache2-mod-php7.0 mariadb-client mariadb-server mtr-tiny nmap php7.0-cli php7.0-curl php7.0-gd php7.0-json php7.0-mcrypt php7.0-mysql php7.0-snmp php7.0-xml php7.0-zip python-memcache python-mysqldb rrdtool snmp snmpd whois curl acl

Add local user for librenms

useradd librenms -d /opt/librenms -M -r
usermod -a -G librenms www-data
systemctl restart mysql

Create user & DB

mysql -uroot -pzaib1234
CREATE DATABASE librenms CHARACTER SET utf8 COLLATE utf8_unicode_ci;
CREATE USER 'librenms'@'localhost' IDENTIFIED BY 'zaib1234';
GRANT ALL PRIVILEGES ON librenms.* TO 'librenms'@'localhost';
FLUSH PRIVILEGES;
exit

Now edit maria-db config file

nano /etc/mysql/mariadb.conf.d/50-server.cnf

& add following

[Within the [mysqld] section please add]

innodb_file_per_table=1
sql-mode=""
lower_case_table_names=0
systemctl restart mysql

Save & Exit.

Web Server Config

Edit PHP files & setup TIMEZONEs

nano /etc/php/7.0/apache2/php.ini
nano /etc/php/7.0/cli/php.ini

Search for ‘date.timezone‘ & change it to

date.timezone = Asia/Karachi

Save & EXit.

Configure Apache

nano /etc/apache2/sites-available/librenms.conf

Add the following config,



DocumentRoot /opt/librenms/html/
ServerName librenms.localhost
AllowEncodedSlashes NoDecode

Require all granted
AllowOverride All
Options FollowSymLinks MultiViews



Save & Exit

a2dissite 000-default
a2ensite librenms.conf
a2enmod rewrite
systemctl restart apache2

Configure snmpd

cp /opt/librenms/snmpd.conf.example /etc/snmp/snmpd.conf

Now edit SNMPD config file and add our SNMP community string like public or other

nano /etc/snmp/snmpd.conf

Edit the text which says RANDOMSTRINGGOESHERE and set your own community string.
save and exit.

curl -o /usr/bin/distro https://raw.githubusercontent.com/librenms/librenms-agent/master/snmp/distro
chmod +x /usr/bin/distro
systemctl restart snmpd

CRONJOBS for librenms

cp /opt/librenms/librenms.nonroot.cron /etc/cron.d/librenms
cp /opt/librenms/misc/librenms.logrotate /etc/logrotate.d/librenms
chown -R librenms:librenms /opt/librenms
setfacl -d -m g::rwx /opt/librenms/rrd /opt/librenms/logs
setfacl -R -m g::rwx /opt/librenms/rrd /opt/librenms/logs

Install LIBRE-NMS via composer command

cd /opt
composer create-project --no-dev --keep-vcs librenms/librenms librenms dev-master

Example:

1- install libre via cmd

Now access libre-nms via browser …

http://yourip/install.php

& follow onscreen steps…

2- libre first step of gui

Make sure to add id password which is authorized to change DB , as required …

 

 

 

 

AT one point, You have to create the config.php manually as showed here.

nano /opt/librenms/config.php

& paste the data as showed in the config window of gui.

<?php
## Have a look in defaults.inc.php for examples of settings you can set here. DO NOT EDIT defaults.inc.php!

### Database config
$config['db_host'] = 'localhost';
$config['db_port'] = '3306';
$config['db_user'] = 'librenms';
$config['db_pass'] = 'zaib1234';
$config['db_name'] = 'librenms';
$config['db_socket'] = '';

// This is the user LibreNMS will run as
//Please ensure this user is created and has the correct permissions to your install
$config['user'] = 'librenms';

### Locations - it is recommended to keep the default
#$config['install_dir'] = "/opt/librenms";

### This should *only* be set if you want to *force* a particular hostname/port
### It will prevent the web interface being usable form any other hostname
#$config['base_url'] = "http://librenms.company.com";

### Enable this to use rrdcached. Be sure rrd_dir is within the rrdcached dir
### and that your web server has permission to talk to rrdcached.
#$config['rrdcached'] = "unix:/var/run/rrdcached.sock";

### Default community
$config['snmp']['community'] = array("public");

### Authentication Model
$config['auth_mechanism'] = "mysql"; # default, other options: ldap, http-auth
#$config['http_auth_guest'] = "guest"; # remember to configure this user if you use http-auth

### List of RFC1918 networks to allow scanning-based discovery
#$config['nets'][] = "10.0.0.0/8";
#$config['nets'][] = "172.16.0.0/12";
#$config['nets'][] = "192.168.0.0/16";

# Update configuration
#$config['update_channel'] = 'release'; # uncomment to follow the monthly release channel
#$config['update'] = 0; # uncomment to completely disable updates 

Save & Exit.

and click on FINISH on GUI.


librenms install done

 

Make sure to remove COMPOSER & set permissions when all is OK. Else it will give error in VALIDATION part.

chown -R librenms:librenms /opt/librenms
apt-get remove composer

 


Some more snaps

  • Add Device

 

add device

librenms add device with snmp

Devices Overview

device view

libre dashboard.JPG

FUN with Mikrotik BRIDGE Series# Redirecting Traffic with Mikrotik Bridge – Part#2

$
0
0

link-redirection

  1. FUN with Mikrotik BRIDGE Series#1. Filter PPPoE Requests – Part#1
  2. FUN with Mikrotik BRIDGE Series# Redirecting Traffic with Mikrotik Bridge – Part#2 < You are Here

Disclaimer! This is important!

This post is related to a solution designed specific to cater some local manipulation requirement therefore you may continue to read it as an reference purpose only !

Every Network is different , so one solution cannot be applied to all. Therefore try to understand logic & create your own solution as per your network scenario. Just dont follow copy paste.

Please donot think that I am an expert on this stuff, I am NOT certified in anything including Mikrotik/Cisco/Linux or Windows. However I have worked with some core networks and I read , research & try stuff all of the time. So I am not speaking/posting about stuff I am formerly trained in, I pretty much go with experience and what I have learned on my own. And , If I don’t know something then I read & learn all about it.

So , please don’t hold me/my-postings to be always 100 percent correct. I am human being , I do make mistakes just like everybody else. However – I do my best, learn from my mistakes and always try to help others

Regard's
Syed Jahanzaib~

Scenario & Requirements:

We want to connect Network A & B using Mikrotik Bridge so that we can transparently intercept some traffic for control & redirection purposes. Example we want to make sure that any dns traffic that is traveling from A to B or B to A should be redirected to Mikrotik DNS for manipulation purposes. Also we would like to Block ICMP traffic travelling between both networks.

Solution:

We are using Mikrotik 2011UiAS-2HnD model.

Port-1 is connected with Network A and Port-2 is connected with Network B.

# BRIDGE Configuration

First we will do Bridge configuration & add ports in it,

/interface bridge
add name=bridge1

/interface bridge port
add bridge=bridge1 interface=ether1
add bridge=bridge1 interface=ether2
/interface bridge settings
set use-ip-firewall=yes

As showed in image below …

bridge rules

# DNS Configuration

Now setup Local DNS server

/ip dns
set allow-remote-requests=yes servers=8.8.8.8

# Now we will add static DNS entry for our requirements
/ip dns static
add address=1.2.3.4 name=aacable.wordpress.com

As showed in image below …

bridge dns add static.JPG

# DNS Redirection

Firewall NAT configuration to redirect DNS traffic travelling via BRIDGE interface to Mikrotik local DNS for manipulation purposes

/ip firewall nat
add action=redirect chain=dstnat comment="Redirect DNS Traffic via BRIDGE to local DNS - Zaib" dst-port=53 in-interface=bridge1 protocol=udp to-ports=53

# ICMP Filteration

Firewall Filter configuration to block ICMP protocol

/ip firewall filter
add action=reject chain=forward comment="Block ICMP Rule in BRIDGE - Zaib" in-interface=bridge1 protocol=icmp reject-with=icmp-network-unreachable

Client Testing

Result of testing NSLOOKUP from user PC. [Before vs After]

bridge - dns resolve nslookup result

Result of testing ICMP & PING from user PC.

bridge - icmp block result result


Linux is amazing 🙂 however Mikrotik is handy most of the times 🙂

Advertisements

Separating NATTING from ROUTING in Mikrotik

$
0
0

nattinv and routing brother.jpg

mikrotik natting and routing

Disclaimer:

Every Network is different , so one solution cannot be applied to all. Therefore try to understand logic & create a solution that can match with your network scenario. Do not follow copy paste blindly.

I am NOT certified in anything Mikrotik/Cisco/Linux or Windows. However I have worked with some core networks and I read & research & try stuff all of the time. So I am not speaking/posting about stuff I am formerly trained in, I pretty much go with experience and what I have learned on my own. And , If I don’t know something then I read & learn all about it.

So , please don’t hold me/my-postings to be always 100 percent correct. I make mistakes just like everybody else. However – I do my best, learn from my mistakes and always try to help others

Regard’s
Syed Jahanzaib


Scenario:

We are using Mikrotik CCR as PPPOE NAS. Its a mix match router where large number of users received private ip (via pppoe connection) and other large portion receives IP from routed Public pool as well.

 


Problem:

When we have network outages like light failure in any particular area , in LOG we can see many PPPoE sessions disconnects with ‘peer not responding‘ messages. Exactly at this moments, our NAS CPU usage reaches to almost 100% , which results in router stops passing any kind of traffic. This can continue for a minute or so on.

As showed in the image below …

pppoe high cpu usage

If you are using Masquarade /NAT on the router, that is the problem. When using Masquarade, RouterOS has to do full connection tracking recalculation on EACH interface connect/disconnect. So if you have lots of PPP session connecting/disconnecting, connection tracking will constantly be recalculated which can cause high CPU usage. When interfaces connect/disconnect, in combination with NAT, it gives you high CPU usage.


Solution OR Possible Workaround :

  • If you have lots of PPP users along with some NATTING rules, Stop using Masquarade on same router that have a lot of dynamic interfaces. DO NOT use NAT on any router that have high number of connecting/disconnecting interfaces , like pppoe/vpn. Place an additional router connected with your PPPoE NAS, and route NAT there.
    Example: Add another router & perform all natting on that router by sending marked traffic from private ip series to that nat router. Setup routing between the PPPoE NAS and the NAT router.

Following is an working example.

1# Main CCR as PPPOE NAS

Interface Details:

  • ETHER1-LAN-: 192.168.88.1/24 < User facing interface where pppoe connections establishes
  • PUBLIC-WAN: 101.11.11.254 < WAN Interface for public IP routing
  • 2-NAT-ROUTER: 192.168.60.2/24  < interface connected with another CCR for natting

PPPoE User IP Pool > 172.16.0.1-172.16.0.255
UPSTREAM ISP Core Router Gateway IP >  101.11.11.36

2# Second CCR as NATTING Router

Interface Details:

  • 2-CCR-LAN: 192.168.60.1/24 < interface connected with main CCR [pppoe]
  • NATTING-WAN: 101.11.11.253 < Wan interface for natting users [traffic coming from main CCR for natting]
  • UPSTREAM ISP Core Router Gateway IP >  101.11.11.36

1# Main CCR Configuration for marking traffic

First we will mark traffic for private/public ip and will create routes for them as well.

/ip firewall mangle
add action=mark-routing chain=prerouting comment="Mark routing for Private IP users - zaib" disabled=no new-routing-mark=nat_routing passthrough=yes src-address=172.16.0.1-172.16.0.255
# We really dont need to mark traffic for public ip's because they will simply pass from our default route , but just for the sake of demonstration we are doing it.
add action=mark-routing chain=prerouting comment="Mark routing for PUBLIC IP users - zaib" disabled=no new-routing-mark=public_routing passthrough=yes src-address=1.1.1.1-1.1.1.255

Make sure you dont have any NAT rule in place. [in NAT section]

Now add Routes for marked traffic

/ip route
add comment="Route private ip traffic via second NAT router" disabled=no distance=1 dst-address=0.0.0.0/0 gateway=192.168.100.1 routing-mark=nat_routing scope=30 target-scope=10
add comment="Route public ip via this router default Gateway" disabled=no distance=1 dst-address=0.0.0.0/0 gateway=101.11.11.36 routing-mark=public_routing scope=30 target-scope=10
# DEFAULT Gateway for router's own traffic - zaib
add check-gateway=ping disabled=no distance=1 dst-address=0.0.0.0/0 gateway=101.11.11.36 scope=30 target-scope=10

Main CCR configuration part is done. Now moving towards second router where all NATTING will be done.

2# NATTING CCR Configuration for Masquerade

First create Default NAT rule [you may want to add ip series for security purposes.

/ip route
add comment="Default Router for NATTING router " disabled=no distance=1 dst-address=0.0.0.0/0 gateway=101.11.11.36 scope=30 target-scope=10
# Add reverse Route so that NATTING router can see the pppoe user directly
add disabled=no distance=1 dst-address=172.16.0.0/16 gateway=192.168.100.2 scope=30 target-scope=10

Testing !

  • Create TEST user in main CCR pppoe NAS,
  • Assign him private ip series profile,
  • Connect this TEST id from test PC & run TRACEROUTE

As showed in the image below …

ccr pppoe active private.JPG

.

nat-vs-route.JPG

RUN Torch on NATTING Router… as we can see that NATTING router is seeing pppoe users directly dueto reverse route in it.

NATTING CCR torch

.


 

Advertisements

Veeam B&R 9.5 Update 3 Error: This Veeam Backup & Replication installation can not be updated automatically

$
0
0

We encountered following error were tried to apply Veeam B&R 9.5 Upgrade Patch 3 .

This Veeam Backup & Replication installation can not be updated automatically. Please contact Veeam customer support for assistance with manual update.

vbr95up3 error.JPG

After some investigation it found that it can occur dueto either you are running trial or if you have problem with the license files. Therefore as a workaround to enforce Update Pack 3 which was required dueto requirement of addition of Esxi 6.5 Vcenter, I followed below steps …

Rename following files

VeeamLicense.dll
[ available in C:\Program Files\Common Files\Veeam\ ]

veComLic.dll
[ available in C:\Program Files\Common Files\Veeam\Backup And Replication ]

Now run the Update Pack 3 and it will run smoothly.

v95up3

 


 

Done.

v br 95 up3 final.JPG

Symantec SEPM 12.x Migration to 14.x

$
0
0

sepm

Today was a hectic day. We received new series of HP G5/8th Gen laptop which supports Windows 10 only  , & when we tried to installed SEPM 12.x client, it stated that the App is not compatible with this version of windows. That was really an hectic news for us because our SEPM server was based on windows 2003 32bit & there is no straightforward method for in-place upgrade to SEPM 14. It requires minimum 64bit of server window 2008 or above server . Any way we managed to migrate SEPM 12.1.6 to SEPM 14.x on Windows 2012 R2 64bit at a cost of whole day brainstorming.

Following are steps I followed for the migration of SEPM v12 W2003 32bit to SEPM v14 on Windows 2012 R2 64bit.


First Take Backup of current SEPM 12.x DB

Step 1: Back up the Database

  1. Click Start Programs Symantec Endpoint Protection Manager > Database Backup and Restore.
  2. Click Back Up. The database backup file name is date_timestamp.zip and is located in the following directory:

\Program Files\Symantec\Symantec Endpoint Protection Manager\data\backup

Copy above folder at some centralized file server folder or USB.

Step 2: Back up the Disaster Recovery File

Copy the following folder at some centralized file server folder or USB.

\Program Files\Symantec\Symantec Endpoint Protection Manager\Server Private Key Backup\

Now shutdown current SEPM Server machine & proceed to install SEPM on new machine


# Install SEPM on new Windows 2012 R2 server

on New Windows 2012 R2 , configure same Computer name & IP Address which was setup on old SEPM server.

Install the Same SEPM version, in my case it was 12.1.6 MP6.

Once installation is done, & When the Management Server Configuration Wizard runs, select Custom configuration & choose Use a recovery file. Point to the backup folder we took in Step 1/2.

As showed in the image below …

1- sav restore from backup.JPG

Once its done, Start the SEPM console one time to confirm if all is running ok.

Restore the database:

To restore DB, Stop the following services,

  • Symantec Endpoint Protection Manager
  • Symantec Endpoint Protection Manager Webserver
  1. Click Start Programs Symantec Endpoint Protection Manager > Database Backup and Restore.
  2. Click Restore.

Once All done, reboot server one time and make sure all services are started properly in the SERVICES console.


User Rights Assignment in Group Policy.

If your SEPM is an member of Active Directory then services will not start dueto lack of LOGON AS SERVICE rights.

As a workaround I installed GROUP POLICY MANAGEMENT on the new SEPM server, and edit Group Policy to add following users in LOGON AS SERVICE section.

  • NT SERVICE\semsrv
  • NT SERVICE\semwebsrv
  • NT SERVICE\SQLANYs_sem5
  • NT SERVICE\semapisrv

Since I was editing Domain Group Policy from the SEPM server itself, that is why I changed Location to local PC and then above accounts was added successfully. As a workaround we can add user SID as well. See following command to get SID of accounts and SID accordingly.

sc showsid semsrv
sc showsid semwebsrv

Client SEPM 14.x Server Console

sepm server.JPG

Client SEPM 14.x Client

client sepm.JPG

Short Notes for Cisco 3850-24T IOS XE Upgrade & Stack Configuration

$
0
0

ZAIB_ Cisco 3850- 24-t.jpg

 


Upgrade Firmware of Cisco 3850 24T-S (INSTALL MODE) by copying FW from TFTP to Switch Flash Directory

Read these first on BUNDLE vs INSTALL mode.

http://blog.qsolved.com/2014/02/what-are-methods-to-boot-and-run-ios-xe.html

http://wannabelab.blogspot.com/2015/09/cisco-catalyst-3650-ios-recovery-via-usb.html

Last week we received new Cisco switches shipment having model WS-C3850-24T which will be replacing existing 3750 stack series in our data center. These switches have 2 methods of booting and running IOS XE software (in 3850).  By default, the switches comes with INSTALL mode. 

Before Upgrading Firmware ,we need to check for current mode in which the switch is currently booted in. [It should be INSTALL mode]

show version | begin Switch Port
Switch Ports Model SW Version SW Image Mode
------ ----- ----- ---------- ---------- ----
* 1 32 WS-C3850-24T 16.3.5b CAT3K_CAA-UNIVERSALK9 INSTALL
Configuration register is 0x102

We can use local USB as well to copy the firmware but since I already have TFTP for centralized backup for all Cisco switches therefore I am using TFTP method in this post.

Having TFTP server will give more leverage of doing various luxuries. copy the latest firmware to TFTP and then from switch console download the switch upgraded ISO and upload it to TFTP root folder. At the time of writing latest IOS XE version for 385024-T-S is cat3k_caa-universalk9.SPA.03.06.08.E.152-2.E8.bin

Connect to switch by ssh/telnet or by console and issue following command

copy tftp: flash:
Address or name of remote host []? 192.168.0.1
Source file name []? cat3k_caa-universalk9.SPA.03.06.08.E.152-2.E8.bin
Destination file name []? cat3k_caa-universalk9.SPA.03.06.08.E.152-2.E8.bin

After some minutes the new IOS will be downloaded to flash directory. in my case it took around 90 seconds to copy 291 MB IOS EX file.

# Switch to enable mode
en
#Issue command to upgrade to new firmware we just downloaded in above steps
request platform software package install switch all file flash:cat3k_caa-universalk9.SPA.03.06.08.E.152-2.E8.bin

Switch will now start the upgrade processing …

--- Starting install local lock acquisition on switch 1 ---
Finished install local lock acquisition on switch 1
Expanding image file: flash:cat3k_caa-universalk9.SPA.03.06.08.E.152-2.E8.bin
[1]: Expanding file
[1]: Finished expanding all-in-one software package in switch 1
SUCCESS: Finished expanding all-in-one software package.
[1]: Performing install
 SUCCESS: install Finished
[1]: install package(s) on switch 1
--- Starting list of software package changes ---
Old files list:
 Removed cat3k_caa-guestshell.16.03.05b.SPA.pkg
 Removed cat3k_caa-rpbase.16.03.05b.SPA.pkg
 Removed cat3k_caa-rpcore.16.03.05b.SPA.pkg
 Removed cat3k_caa-srdriver.16.03.05b.SPA.pkg
 Removed cat3k_caa-wcm.16.03.05b.SPA.pkg
 Removed cat3k_caa-webui.16.03.05b.SPA.pkg
New files list:
 Added cat3k_caa-base.SPA.03.06.08E.pkg
 Added cat3k_caa-drivers.SPA.03.06.08E.pkg
 Added cat3k_caa-infra.SPA.03.06.08E.pkg
 Added cat3k_caa-iosd-universalk9.SPA.152-2.E8.pkg
 Added cat3k_caa-platform.SPA.03.06.08E.pkg
 Added cat3k_caa-wcm.SPA.10.2.180.0.pkg
Finished list of software package changes
SUCCESS: Software provisioned. New software will load on reboot.
[1]: Finished install successful on switch 1
Checking status of install on [1]
[1]: Finished install in switch 1
SUCCESS: Finished install: Success on [1]

Now issue reload command to restart/reload the new config.

reload

Continue to reboot.

After the upgrade We will see new version in action !

show version | b SW
Switch Ports Model SW Version SW Image Mode
------ ----- ----- ---------- ---------- ----
* 1 32 WS-C3850-24T 03.06.08E cat3k_caa-universalk9 INSTALLConfiguration
register is 0x102


Short Notes on STACK Configuration!

cisco 3850 stack cable.jpg

Stacking is simple ! Example we want to connect 2 switches in stack mode.

1.  Power up the switch master only, let it boot completely,

2.   Attach the stacking cable from 1st switch to second one,

3.  Power up the second switch.

By default there will be no extra config required. and if both switches matches they will be added in STACK automatically.

JUST MAKE SURE THAT …

1.  All switches are running the same IOS version, same Feature Set (example IP Base);

Setting Switch Priority for MASTER

We can set switch priority to decide which switch should be MASTER. To do this, Power ON the Master switch only and issue following command

sh switch

Switch/Stack Mac Address : 009a.d29c.daXX - Local Mac Address
Mac persistency wait time: Indefinite
H/W Current
Switch# Role Mac Address Priority Version State
------------------------------------------------------------
*1 Active 009a.d29c.daXX 15 V07 Ready

Note down your switch number.

# switch to ENABLE mode
en
# Set Switch Priority
switch 1 priority 15
# WRITE CONFIG
wr
# RELOAD the configuration / reboot
reload

now boot second switch.

Once all Done. Issue following command to confirm the STACKING status.

Switch#sh switch
Switch/Stack Mac Address : 009a.d29c.da00 - Local Mac Address
Mac persistency wait time: Indefinite
H/W Current
Switch# Role Mac Address Priority Version State
------------------------------------------------------------
*1 Active 009a.d29c.da00 15 V07 Ready
2 Standby 682c.7b3c.4180 1 V07 Ready

Check Stack Ports status

Switch#show switch stack-ports
Switch# Port1 Port2
----------------------------
1 OK DOWN
2 OK DOWN

Some Useful STACK commands

show license right-to-use
switch stack-bandwidth
show switch detail

Following is one good Link for 3750 stack configuration …

https://www.petenetlive.com/KB/Article/0001205



Regard’s
Syed Jahanzaib ~

Advertisements

Vmware ESXI Upgrade Path Notes

$
0
0

Few days back, we did major upgrades in our data center including esxi upgrades and some network related rack replacement. We were running few vmware esxi 5.0 servers along with vcenter 5.0. To upgrade all servers with 6.5 , we follows below path …

  • Added new 6.5v Vcenter * recommended*
    [OR upgrade existing 5.x Vcenter to 6.5 VC]
  • Now Upgrade each esxi server to 5.5 one by one
  • Now Upgrade each esxi server to 6.5 one by one
  • Now add all esxi servers into new VC 6.5 & all Done 🙂

I had the esxi ISO in CD, therefore I shutdown the all VM guests, and then …

  • Rebooted the esxi server from the ESXI ISO CD
  • It auto detect the previously install esxi, and I just selected UPGRADE

& everything went smoothly.


VIB error when upgrade from 5.5 to 6.5

esxi vib error

When I tried to upgrade from 5.5 to 6.5 it gave some VIB errors as showed in the image above. (It was taken with nokia 3310 🙂

To resolve conflicting VIB, I enabled SSH on the esxi host, logged in via ssh, and issueed following

~ # esxcli software vib list |grep Broad

brcm 550.2.5.11-000000 Broadcom VMwareAccepted 2018-05-26
misc-cnic-register 1.78.75.v55.1-1OEM.550.0.0.1331820 Broadcom VMwareCertified 2018-05-26
net-bnx2 2.2.4f.v55.3-1OEM.550.0.0.1331820 Broadcom VMwareCertified 2018-05-26
net-bnx2x 1.78.80.v55.3-1OEM.550.0.0.1331820 Broadcom VMwareCertified 2018-05-26
net-cnic 1.78.76.v55.7-1OEM.550.0.0.1331820 Broadcom VMwareCertified 2018-05-26
net-tg3 3.135b.v55.1-1OEM.550.0.0.1331820 Broadcom VMwareCertified 2018-05-26
scsi-bnx2fc 1.78.76.v55.1-1OEM.550.0.0.1331820 Broadcom VMwareCertified 2018-05-26
scsi-bnx2i 2.78.76.v55.1-1OEM.550.0.0.1331820 Broadcom VMwareCertified 2018-05-26

# In my case the conflicting driver was net-bnx2x so I just removed it using below command,

~ # esxcli software vib remove -n net-bnx2x

Exit, & Rebooted, afterwards upgrade the esxi 6.5 worked well.


Datastore Conflicts with an existing datacenter that has the same URL

When I removed one esxi from the VC 6.5 & and tried to add it again, it gave following error

olddatastorage.JPG

After inspection, it came to my knowledge that one of guest from another esxi server is using this data store , I simply moves this to another esxi, and then the host got added successfully !


Regard’s
Syed JahanzaiB

Viewing all 409 articles
Browse latest View live