NETCONF, ncclient and Network Automation

I my last blog post I had discussed various network management protocols and promised that I will try to experiment with ncclient. ncclient is a Python library you need as a NETCONF client to communicate with a NETCONF server in this instance it will be an instance of Cisco CSR1000v.

The whole process is very straight forward.. in brief

  1. A Ubuntu Server instance ( could also be a Windows server/Workstation)
  2. Python install and configured (all instances of Ubuntu server have Python installed by default)
  3. ncclient installed and configured
  4.  NETCONF enabled on network device (HP, Cisco, Juniper and other supported)

My not so beefy laptop (8GB RAM and intel i5) but powerful enough to run VMware WorkStation with Ubuntu Server and Cisco CSR1000v instance

CSR1000V  VMware Workstation Settings


The Network Adapter corresponds to the GigabitEthernet1 Interface on the router

interface GigabitEthernet1
ip address
negotiation auto

Enable SSH and NETCONF access on the router

TEST_CSR#conf t
TEST_CSR(config)# ip ssh rsa keypair-name sshkeys
TEST_CSR(config)# crypto key generate rsa usage-keys label sshkeys modulus 1024
TEST_CSR(config)# ip ssh timeout 120
TEST_CSR(config)# ip ssh version 2

TEST_CSR(config)# netconf ssh

Ubuntu server NIC interface configured with an address of

Ping from the server – Success !!!

The next item on the agenda was to set up the ncclient.  Please follow the well explained instructions documented here.  Thanks for your help guys.

Cisco CSR1000v, and ncclient setup completed…

Get Device Config

>>> cisco_manager = manager.connect(host='',
... port=22,
... username='cisco',
... password='cisco',
... hostkey_verify=False,
... device_params={'name': 'csr'},
... allow_agent=False,
... look_for_keys=False
... )

Connection established.

RPC call requesting running configuration

>>> c = cisco_manager.get_config(source='running')

Print output on terminal console

>>> c

<?xml version=”1.0″ encoding=”UTF-8″?><rpc-reply message-id=”urn:uuid:e147c6d6-cebf-11e5-afac-000c29e6c046″ xmlns=”urn:ietf:params:netconf:base:1.0″>

! Last configuration change at 23:55:29 UTC Mon Feb 8 2016
version 15.4
service timestamps debug datetime msec
service timestamps log datetime msec
service password-encryption
no platform punt-keepalive disable-kernel-core
platform console virtual
hostname TEST_CSR




Small step hopefully in right direction.. I will try few other NETCONF options .. stay tuned.



SNMP, NETCONF, YANG, RESTCONF and Network Automation

In the past life as a network engineer, when working on a large deployment, I have relied on the Excel spreadsheet with macros to generate configuration files.  Copy and Paste – Job done.  Any change in the script, I had to re-run the whole process and trust me it was not fun.

Network monitoring software like HP OpenView and CiscoWorks, offered limited functionality around automation as they were primarily used for monitoring and configuration management.  Depending on the complexity of the change to be applied, using a combination of UNIX shell and Python scripts I have managed to get my way around.

I have always wanted to understand what was missing in the current management tools and what is coming down the line and now that every other technology is now a Software Defined (SD) something including Network, WAN, DC..its time that I dig deeper.  This blog will be an overview of the technologies and follow up getting started with NETCONF and YANG in the next few week.


A Simple Network Management  Protocol (SNMP) that can be used to configure and retrieve variables on a device.  MIB – A Management Information Base that defines all data managed by SNMP.

  • Transport over UDP
  • Stateless
  • Unable to backup and restore element
  •  Operations
    • GET
    • GET-NEXT
    • SET
    • TRAP ….

SNMP has been around for a long time and can also be used to configure variable on a device.  To Add VLAN requires a quite a few steps, and you would think, it will be easier to login to each switch/router and make that change and maybe this complexity is the reason why SNMP is rarely used for configuring devices.

A comprehensive list of shortcomings with SNMP discussed at the Network Management workshop in 2002 can be found here SNMP Framework Discussion


RFC 4741  The Network Configuration Protocol (NETCONF) provides mechanisms to install, manipulate, and delete the configuration of network devices. It uses an Extensible Markup Language (XML)-based data encoding for the configuration data as well as the protocol messages. The NETCONF protocol operations are realized on top of a simple Remote Procedure Call (RPC) layer.

  • Designed to be as a replacement for CLI
  • Transported over SSH (TCP)  and stateless
  • Service and Network management
  • Supports network wide transactions
  • Consistency check built in
  • Named data stores ex – Startup, Running, File/URL’s
  • Distinction between configuration and state data
  • XML Payload – Modelled in YANG

NETCONF can be conceptually partitioned into four layers:



As defined in the RFC 6020 YANG is a data modeling language used to model configuration and state data manipulated by the Network Configuration Protocol (NETCONF), NETCONF remote procedure calls, and NETCONF notifications.

YANG is a language used to model data for the NETCONF protocol. A YANG module defines a hierarchy of data that can be used for NETCONF- based operations, including configuration, state data, Remote Procedure Calls (RPCs), and notifications. This allows a complete description of all data sent between a NETCONF client and server

YANG is to NETCONF as MIB is to SNMP.

Next blog will be an attempt to setup ncclient and extract configuration from a network device. Fingers crossed..


RESTCONF is still a “work in progress” as defined “draft-ietf-netconf-restconf-05”.  

The NETCONF protocol defines configuration datastores and a set of Create, Retrieve, Update, Delete (CRUD) operations that can be used to access these datastores. The YANG language defines the syntax and semantics of datastore content, operational data, protocol operations, and event notifications. RESTCONF uses HTTP operations to provide CRUD operations on a NETCONF datastore containing YANG defined data

The one good point in and when its is standard if have some knowledge of HTTP operations this quote from the text will bring a smile “Since NETCONF protocol operations are not relevant,the user should not need any prior knowledge of NETCONF in order to use RESTCONF”

Both in Open Source world with OpenDaylight Foundation controller and Cisco commercial Cisco Open SDN Controller, NETCONF and RESTCONF are well on it way being integral part of the offerings.  Time will tell on as to when these products are widely adopted.




Getting started with Network Automation using Python and Netmiko – part 2

Let’s get started where we left off from part one.. With an upstream and downstream reading from the router, rather crudely extracted from a line of text output and placed into a pair of variables.

So, previously we were able to print the bandwidth values prepended by some text. This time we will wrap some text around these variables to create meaningful IOS commands ready for delivery to our device.

Let’s start where we generated screen output from last time :

print ("bandwidth",upstream)
print ("bandwidth receive",downstream)

We can use this text and the variables to create a new string to build each command we’ll be sending to the router :

upstream_cli = "bandwidth " + upstream
downstream_cli = "bandwidth receive " + downstream

Now we need to make some preparations to send these to the router by placing them into a list.

cli_commands = ['interface dialer 0', upstream_cli, downstream_cli]

We have created a list, which Netmiko will iterate over and deliver to the router in order. We get this done by passing the list to Netmiko’s send_config_set method. Remember that we already setup the ssh connection via net_connect to the router earlier in part 1, which should still be active.

Recall that Netmiko already knows we are configuring a Cisco device, so it helpfully adds the required ‘conf t’ as it delivers the commands for us.

net_connect.enable() # enter enable mode
net_connect.send_config_set(cli_commands) # send the commands from our list
net_connect.disconnect() # close the connection gracefully

And there we have it, our first automated configuration change is complete. If we log onto the router, a ‘show run int dial 0’ verifies that our commands have been successfully delivered.

interface Dialer0
bandwidth 760
bandwidth receive 4347

Hopefully this has been of some use. I’ll be covering REST API calls using Python and the Cisco CSR1000V in the near future so keep an eye on my twitter feed @nu110 for updates.

For reference, here is the finished Python code :

from netmiko import ConnectHandler

router = {
'device_type': 'cisco_ios',
'ip': 'IP_ADDRESS',
'username': 'USERNAME',
'password': 'PASSWORD',
'port' : 22, # optional, defaults to 22
'secret': 'ENABLE PASSWORD', # optional, defaults to ''
'verbose': True, # optional, defaults to True

net_connect = ConnectHandler(**router)

output = net_connect.send_command('show controllers vdsl 0')
output = output.split('\n')

print ("parsing output..\n")
for line in output:
    if '(kbps)' in line:
        speed = line
        print (line,'\n')

speed = speed.split(' ')
downstream = str(speed[19])
upstream = str(speed[37])

upstream_cli = "bandwidth " + upstream
downstream_cli = "bandwidth receive " + downstream
cli_commands = ['interface dialer 0', upstream_cli, downstream_cli]

for line in cli_commands:
    print (line)


Getting started with Network Automation using Python and Netmiko

Keen to get started on my automation journey, I struggled for a long time with telnet libraries like telnetlib and even resorted to dropping into an SSH process and scraping output via pexpect. To say it was cumbersome and error-prone is an understatement. I eventually gave up and went back to CLI bashing and using Kiwi / Solarwinds Cattools for delivering bulk changes.

A couple of weeks ago however I made a discovery, somebody much smarter than I am figured out a clean and reliable way to connect to network devices and return output. The good news is they kindly built a Python library to make life much easier for the rest of us. It’s not all about screen scraping output either, this library does a great job of delivering configuration changes as well.

Netmiko extends Paramiko (a Python library for managing SSH connections) by adding support for network devices and is written by Kirk Byers. As with all good open source projects, source code can be checked out from GitHub, or installed easily using Python’s pip installer.

I opted for the pip install on OSX, which is incredibly simple :

$pip install netmiko
Collecting netmiko
  Downloading netmiko-0.2.5-py2.py3-none-any.whl
Installing collected packages: netmiko
Successfully installed netmiko-0.2.5

There are several ways to install pip on OSX, some examples can be found on StackExchange (

To get started with a basic example, load up the Python interpreter and import the Netmiko libary :

Python 2.7.10 (v2.7.10:15c95b7d81dc, May 23 2015, 09:33:12)
[GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from netmiko import ConnectHandler

From here we can easily connect to a Cisco IOS device (or Juniper Junos, Dell Force10, Fortinet etc) and collect some basic DSL line statistics :

router = {
'device_type': 'cisco_ios',
'ip': '',
'username': 'USERNAME',
'password': 'PASSWORD',
'port' : 22, # optional, defaults to 22
'secret': 'ENABLE', # optional, defaults to ''
'verbose': True, # optional, defaults to True

net_connect = ConnectHandler(**router)

output = net_connect.send_command('show controllers vdsl 0')
output = output.split('\n')
#output is returned as a single string, so let's split this into lines in a list for iteration later

for line in output:
    if '(kbps)' in line:
    speed = line
    print (line,'\n')
    #for debugging purposes, let us know when we see the (kbps) line

If we separate whitespace on the line we’ve found, we can locate the upstream and downstream speeds :

speed = speed.split(' ')
downstream = str(speed[19]) #list entry 19 = downstream speed
upstream = str(speed[37]) #list entry 37 = upstream speed
print ("bandwidth",upstream)
print ("bandwidth receive",downstream)

Output from this script :

$ python3
SSH connection established to
Interactive SSH session established
parsing output..

Speed (kbps): 0 4347 0 760

bandwidth 760
bandwidth receive 4347

There are probably neater and more Pythonic ways to achieve this, but I’m just starting out on my journey and I’m sure my code will improve over time.

That aside, it would be trivial take these variables and use them to generate some configuration, dropping them back onto the relevant dialer interface using the net_connect.send_config_set method, helping monitoring tools like Solarwinds Orion report the correct interface speeds for DSL interfaces, rather than the defaults we usually see. I’ll save that for my next post though..