NETCONF, ncclient and Network Automation

I my last blog post I had discussed various network management protocols and promised that I will try to experiment with ncclient. ncclient is a Python library you need as a NETCONF client to communicate with a NETCONF server in this instance it will be an instance of Cisco CSR1000v.

The whole process is very straight forward.. in brief

  1. A Ubuntu Server instance ( could also be a Windows server/Workstation)
  2. Python install and configured (all instances of Ubuntu server have Python installed by default)
  3. ncclient installed and configured
  4.  NETCONF enabled on network device (HP, Cisco, Juniper and other supported)

My not so beefy laptop (8GB RAM and intel i5) but powerful enough to run VMware WorkStation with Ubuntu Server and Cisco CSR1000v instance

CSR1000V  VMware Workstation Settings

Capture

The Network Adapter corresponds to the GigabitEthernet1 Interface on the router

interface GigabitEthernet1
ip address 192.168.75.1 255.255.255.0
negotiation auto

Enable SSH and NETCONF access on the router

TEST_CSR#conf t
TEST_CSR(config)# ip ssh rsa keypair-name sshkeys
TEST_CSR(config)# crypto key generate rsa usage-keys label sshkeys modulus 1024
TEST_CSR(config)# ip ssh timeout 120
TEST_CSR(config)# ip ssh version 2


TEST_CSR(config)# netconf ssh

Ubuntu server NIC interface configured with an address of 192.168.75.128/24.

Ping 192.168.75.1 from the server – Success !!!

The next item on the agenda was to set up the ncclient.  Please follow the well explained instructions documented here.  Thanks for your help guys.

Cisco CSR1000v, and ncclient setup completed…

Get Device Config

>>> cisco_manager = manager.connect(host='192.168.75.1',
... port=22,
... username='cisco',
... password='cisco',
... hostkey_verify=False,
... device_params={'name': 'csr'},
... allow_agent=False,
... look_for_keys=False
... )

Connection established.

RPC call requesting running configuration

>>> c = cisco_manager.get_config(source='running')

Print output on terminal console

>>> c

<?xml version=”1.0″ encoding=”UTF-8″?><rpc-reply message-id=”urn:uuid:e147c6d6-cebf-11e5-afac-000c29e6c046″ xmlns=”urn:ietf:params:netconf:base:1.0″>

! Last configuration change at 23:55:29 UTC Mon Feb 8 2016
!
version 15.4
service timestamps debug datetime msec
service timestamps log datetime msec
service password-encryption
no platform punt-keepalive disable-kernel-core
platform console virtual
!
hostname TEST_CSR
!
boot-start-marker
boot-end-marker

…..truncated

</cli-config-data-block></data></rpc-reply>

 

Small step hopefully in right direction.. I will try few other NETCONF options .. stay tuned.

@hkdaiya

 

SNMP, NETCONF, YANG, RESTCONF and Network Automation

In the past life as a network engineer, when working on a large deployment, I have relied on the Excel spreadsheet with macros to generate configuration files.  Copy and Paste – Job done.  Any change in the script, I had to re-run the whole process and trust me it was not fun.

Network monitoring software like HP OpenView and CiscoWorks, offered limited functionality around automation as they were primarily used for monitoring and configuration management.  Depending on the complexity of the change to be applied, using a combination of UNIX shell and Python scripts I have managed to get my way around.

I have always wanted to understand what was missing in the current management tools and what is coming down the line and now that every other technology is now a Software Defined (SD) something including Network, WAN, DC..its time that I dig deeper.  This blog will be an overview of the technologies and follow up getting started with NETCONF and YANG in the next few week.

SNMP

A Simple Network Management  Protocol (SNMP) that can be used to configure and retrieve variables on a device.  MIB – A Management Information Base that defines all data managed by SNMP.

  • Transport over UDP
  • Stateless
  • Unable to backup and restore element
  •  Operations
    • GET
    • GET-NEXT
    • SET
    • TRAP ….

SNMP has been around for a long time and can also be used to configure variable on a device.  To Add VLAN requires a quite a few steps, and you would think, it will be easier to login to each switch/router and make that change and maybe this complexity is the reason why SNMP is rarely used for configuring devices.

A comprehensive list of shortcomings with SNMP discussed at the Network Management workshop in 2002 can be found here SNMP Framework Discussion

NETCONF

RFC 4741  The Network Configuration Protocol (NETCONF) provides mechanisms to install, manipulate, and delete the configuration of network devices. It uses an Extensible Markup Language (XML)-based data encoding for the configuration data as well as the protocol messages. The NETCONF protocol operations are realized on top of a simple Remote Procedure Call (RPC) layer.

  • Designed to be as a replacement for CLI
  • Transported over SSH (TCP)  and stateless
  • Service and Network management
  • Supports network wide transactions
  • Consistency check built in
  • Named data stores ex – Startup, Running, File/URL’s
  • Distinction between configuration and state data
  • XML Payload – Modelled in YANG

NETCONF can be conceptually partitioned into four layers:

blog

YANG

As defined in the RFC 6020 YANG is a data modeling language used to model configuration and state data manipulated by the Network Configuration Protocol (NETCONF), NETCONF remote procedure calls, and NETCONF notifications.

YANG is a language used to model data for the NETCONF protocol. A YANG module defines a hierarchy of data that can be used for NETCONF- based operations, including configuration, state data, Remote Procedure Calls (RPCs), and notifications. This allows a complete description of all data sent between a NETCONF client and server

YANG is to NETCONF as MIB is to SNMP.

Next blog will be an attempt to setup ncclient and extract configuration from a network device. Fingers crossed..

RESTCONF

RESTCONF is still a “work in progress” as defined “draft-ietf-netconf-restconf-05”.  

The NETCONF protocol defines configuration datastores and a set of Create, Retrieve, Update, Delete (CRUD) operations that can be used to access these datastores. The YANG language defines the syntax and semantics of datastore content, operational data, protocol operations, and event notifications. RESTCONF uses HTTP operations to provide CRUD operations on a NETCONF datastore containing YANG defined data

The one good point in and when its is standard if have some knowledge of HTTP operations this quote from the text will bring a smile “Since NETCONF protocol operations are not relevant,the user should not need any prior knowledge of NETCONF in order to use RESTCONF”

Both in Open Source world with OpenDaylight Foundation controller and Cisco commercial Cisco Open SDN Controller, NETCONF and RESTCONF are well on it way being integral part of the offerings.  Time will tell on as to when these products are widely adopted.

 

 

 

Getting started with Network Automation using Python and Netmiko – part 2

Let’s get started where we left off from part one.. With an upstream and downstream reading from the router, rather crudely extracted from a line of text output and placed into a pair of variables.

So, previously we were able to print the bandwidth values prepended by some text. This time we will wrap some text around these variables to create meaningful IOS commands ready for delivery to our device.

Let’s start where we generated screen output from last time :

print ("bandwidth",upstream)
print ("bandwidth receive",downstream)

We can use this text and the variables to create a new string to build each command we’ll be sending to the router :

upstream_cli = "bandwidth " + upstream
downstream_cli = "bandwidth receive " + downstream

Now we need to make some preparations to send these to the router by placing them into a list.

cli_commands = ['interface dialer 0', upstream_cli, downstream_cli]

We have created a list, which Netmiko will iterate over and deliver to the router in order. We get this done by passing the list to Netmiko’s send_config_set method. Remember that we already setup the ssh connection via net_connect to the router earlier in part 1, which should still be active.

Recall that Netmiko already knows we are configuring a Cisco device, so it helpfully adds the required ‘conf t’ as it delivers the commands for us.

net_connect.enable() # enter enable mode
net_connect.send_config_set(cli_commands) # send the commands from our list
net_connect.disconnect() # close the connection gracefully

And there we have it, our first automated configuration change is complete. If we log onto the router, a ‘show run int dial 0’ verifies that our commands have been successfully delivered.

interface Dialer0
bandwidth 760
bandwidth receive 4347
-snip-

Hopefully this has been of some use. I’ll be covering REST API calls using Python and the Cisco CSR1000V in the near future so keep an eye on my twitter feed @nu110 for updates.

For reference, here is the finished Python code :

from netmiko import ConnectHandler

router = {
'device_type': 'cisco_ios',
'ip': 'IP_ADDRESS',
'username': 'USERNAME',
'password': 'PASSWORD',
'port' : 22, # optional, defaults to 22
'secret': 'ENABLE PASSWORD', # optional, defaults to ''
'verbose': True, # optional, defaults to True
}

net_connect = ConnectHandler(**router)

output = net_connect.send_command('show controllers vdsl 0')
output = output.split('\n')

print ("parsing output..\n")
for line in output:
    if '(kbps)' in line:
        speed = line
        print (line,'\n')

speed = speed.split(' ')
downstream = str(speed[19])
upstream = str(speed[37])

upstream_cli = "bandwidth " + upstream
downstream_cli = "bandwidth receive " + downstream
cli_commands = ['interface dialer 0', upstream_cli, downstream_cli]

for line in cli_commands:
    print (line)

net_connect.enable()
net_connect.send_config_set(cli_commands)
net_connect.disconnect()

Getting started with Network Automation using Python and Netmiko

Keen to get started on my automation journey, I struggled for a long time with telnet libraries like telnetlib and even resorted to dropping into an SSH process and scraping output via pexpect. To say it was cumbersome and error-prone is an understatement. I eventually gave up and went back to CLI bashing and using Kiwi / Solarwinds Cattools for delivering bulk changes.

A couple of weeks ago however I made a discovery, somebody much smarter than I am figured out a clean and reliable way to connect to network devices and return output. The good news is they kindly built a Python library to make life much easier for the rest of us. It’s not all about screen scraping output either, this library does a great job of delivering configuration changes as well.

Netmiko extends Paramiko (a Python library for managing SSH connections) by adding support for network devices and is written by Kirk Byers. As with all good open source projects, source code can be checked out from GitHub, or installed easily using Python’s pip installer.

I opted for the pip install on OSX, which is incredibly simple :

$pip install netmiko
Collecting netmiko
  Downloading netmiko-0.2.5-py2.py3-none-any.whl
Installing collected packages: netmiko
Successfully installed netmiko-0.2.5

There are several ways to install pip on OSX, some examples can be found on StackExchange (https://stackoverflow.com/questions/17271319/installing-pip-on-mac-os-x).

To get started with a basic example, load up the Python interpreter and import the Netmiko libary :

$python
Python 2.7.10 (v2.7.10:15c95b7d81dc, May 23 2015, 09:33:12)
[GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from netmiko import ConnectHandler

From here we can easily connect to a Cisco IOS device (or Juniper Junos, Dell Force10, Fortinet etc) and collect some basic DSL line statistics :

router = {
'device_type': 'cisco_ios',
'ip': '1.2.3.4',
'username': 'USERNAME',
'password': 'PASSWORD',
'port' : 22, # optional, defaults to 22
'secret': 'ENABLE', # optional, defaults to ''
'verbose': True, # optional, defaults to True
}

net_connect = ConnectHandler(**router)

output = net_connect.send_command('show controllers vdsl 0')
output = output.split('\n')
#output is returned as a single string, so let's split this into lines in a list for iteration later

for line in output:
    if '(kbps)' in line:
    speed = line
    print (line,'\n')
    #for debugging purposes, let us know when we see the (kbps) line

If we separate whitespace on the line we’ve found, we can locate the upstream and downstream speeds :

speed = speed.split(' ')
downstream = str(speed[19]) #list entry 19 = downstream speed
upstream = str(speed[37]) #list entry 37 = upstream speed
print ("bandwidth",upstream)
print ("bandwidth receive",downstream)

Output from this script :

$ python3 dsl.py
SSH connection established to 1.2.3.4:22
Interactive SSH session established
parsing output..

Speed (kbps): 0 4347 0 760

bandwidth 760
bandwidth receive 4347

There are probably neater and more Pythonic ways to achieve this, but I’m just starting out on my journey and I’m sure my code will improve over time.

That aside, it would be trivial take these variables and use them to generate some configuration, dropping them back onto the relevant dialer interface using the net_connect.send_config_set method, helping monitoring tools like Solarwinds Orion report the correct interface speeds for DSL interfaces, rather than the defaults we usually see. I’ll save that for my next post though..

Linux IP clustering with ucarp

A quick way to configure a cluster / failover address between two (or more) Ubuntu / Linux servers.

Install the ucarp package (I originally looked at using VRRP, however noted PFSense used ucarp so gave that a try and stuck with it) :

sudo apt-get install ucarp

edit /etc/network/interfaces:

auto eth0
iface eth0 inet static
address 10.10.10.2
netmask 255.255.255.0
network 10.10.10.0
broadcast 10.10.10.255
gateway 10.10.10.254
dns-nameservers 8.8.8.8

# UCARP cluster IP config
ucarp-vid 101
ucarp-vip 10.10.10.1
ucarp-password YOUR-PASSWORD
ucarp-advskew 10
ucarp-advbase 1
ucarp-master no

iface eth0:ucarp inet static
address 10.10.10.1
netmask 255.255.255.0

Physical cable test on a Cisco switch

Here’s how to run a TDR (Time Domain Reflection) cable test on a Cisco Catalyst switch.

This should work on the 2960, 3750 and 4500 range as far as I’m aware.

To run the test :

switch# test cable-diagnostics tdr interface gig 1/0/1
 TDR test started on interface Gi1/0/1
 A TDR test can take a few seconds to run on an interface
 Use 'show cable-diagnostics tdr' to read the TDR results.

Results from a normal cable run look like this :

switch# show cable-diagnostics tdr int gig 1/0/2
Interface Speed Local pair Pair length Remote pair Pair status
--------- ----- ---------- ------------------ ----------- --------------------
Gi1/0/2  1000M Pair A 64 +/- 10 meters Pair B Normal
               Pair B 64 +/- 10 meters Pair A Normal
               Pair C 64 +/- 10 meters Pair D Normal
               Pair D 64 +/- 10 meters Pair C Normal

And from a faulty cable run something like this :

switch# show cable-diagnostics tdr int gig 1/0/1
 Interface Speed Local pair Pair length Remote pair Pair status
 --------- ----- ---------- ------------------ ----------- --------------------
 Gi1/0/1 auto Pair A 0 +/- 10 meters N/A Open
              Pair B 0 +/- 10 meters N/A Short/Crosstalk
              Pair C 1 +/- 10 meters N/A Short/Crosstalk
              Pair D 63 +/- 10 meters N/A Open

Read more here : https://supportforums.cisco.com/blog/9913546/switch-how-test-cable-status

 

Cisco 3G router basic config

Here is the basis of a Cisco 3G router config, from an 887VAG. Includes the common UK carrier APN settings.

O2 contract :
cellular 0 gsm profile create 1 mobile.o2.co.uk

O2 PayG :
cellular 0 gsm profile create 1 payandgo.o2.co.uk

Vodafone contract :
cellular 0 gsm profile create 1 internet

Three PayG :
cellular 0 gsm profile create 1 three.co.uk

Lebara PayG :
cellular 0 gsm profile create 1 uk.lebara.mobi

General config

chat-script gsm "" "AT!SCACT=1,1" TIMEOUT 60 "OK"

interface Cellular0
ip address negotiated
encapsulation slip
dialer in-band
dialer idle-timeout 0
dialer string gsm
dialer-group 1
async mode interactive

dialer-list 1 protocol ip permit

ip route 0.0.0.0 0.0.0.0 cellular 0

EEM Script to shut / no shut ATM interface automatically

Tired of DSL lines mysteriously dropping I knocked together this EEM script to bounce the ATM interface on a Cisco router if it has been down for 1 minute. Known affectionately as the “ATM brown trousers” script 🙂

This is a dirty hack, but sometimes needs must, especially when you’re at the end of a long DSL line that just never settles.

track 1 interface ATM0 line-protocol
delay down 60 up 5

event manager applet atm0-down
event track 1 state down
action 1.0 syslog msg “%ATM-BRWNTRSRS: Interface ATM 0 failed, reset via EEM.”
action 1.1 cli command “enable”
action 1.2 cli command “conf t”
action 1.3 cli command “interface atm 0”
action 1.4 cli command “shut”
action 1.5 wait 5
action 1.6 cli command “no shut”
action 1.7 cli command “end”
action 1.8 syslog msg “EEM script complete”
action 1.9 wait 60
action 2.0 snmp-trap strdata “%ATM-BRWNTRSRS: Interface ATM 0 failed, reset via EEM.”