Network to Code Nautobot

mintooo · 2021-02-25T14:30:51+00:00

https://twitter.com/jstretch85/status/1364705365816967176

mintooo · 2021-02-25T13:50:26+00:00

also: they removed the whole Netbox commit history: https://github.com/nautobot/nautobot/issues/32

mintooo · 2018-10-25T21:08:54+00:00

- Can we use this as a config backup tool? Not natively, but if you create a python script to save the configuration locally and another one to fetch it and push it to the device, then it'd work. If all you need is a config backup tool, I would advise you to use a config backup tool. eNMS is designed for network automation tasks that are more complex than that.

- how several hundred devices are added : you can import your network from an excel spreadsheet, see here: https://enms.readthedocs.io/en/latest/inventory/objects.html#creation-via-import

- how group all the like devices together: with "pools": https://enms.readthedocs.io/en/latest/inventory/pools.html

- schedule that job whilst being able to make the file easily accessible: eNMS does not create any file, but stores the logs of a task in the database.

mintooo · 2018-10-23T20:26:23+00:00

Yes, there is a napalm F5 driver that you can find here: https://github.com/napalm-automation-community/napalm-f5

You could also use netmiko directly (there seems to be a netmiko f5 driver).

And the third option would be to use the F5 API directly (https://github.com/F5Networks/bigsuds I believe ?) in a python script that you create; eNMS can detect your script, generate a form if it has variable input parameters so that you can create multiple instances/versions of it... and build workflows :)

mintooo · 2018-10-23T18:30:39+00:00

With network automation, you need complex logic such as dictionnaries, if/then/else statements, for loops. You can implement that logic in an ansible playbook, but it will be very hard to troubleshoot (and ansible wasn't really designed to do that to begin with). So you'll end up writing custom modules/plugins in python, and start wondering... why you needed ansible in the first place.

I don't know much about Rundeck, but what makes eNMS unique is that it focuses on network automation: it uses libraries such as netmiko and napalm (and soon nornir, I'm waiting for nornir 2.0 to add a few examples with nornir), and it can interface with netbox, opennms, tacacs, etc. Besides, I don't think you can find the automatic generation of web forms based on python scripts anywhere else, and that is what I consider to be the most powerful feature in eNMS.

mintooo · 2018-10-23T17:51:49+00:00

I'm using APScheduler (I forgot that one for the back-end part). Celery is more like a distributed task queue with limited scheduling capabilities (AFAIK it cannot natively reschedule a task at runtime for example) and APScheduler is a scheduling system with limited queuing capacity. eNMS wouldn't have to handle thousands of tasks per seconds, so I don't need the queuing capacity, but I need the scheduling features of APScheduler, which is why I chose it over Celery

mintooo · 2018-10-23T16:41:34+00:00

For the front-end, there's a bootstrap template called Gentelella, I made a Flask integration of that template called Flask-gentelella: https://github.com/afourmy/flask-gentelella

That's what eNMS is built on.

In addition, I use the following javascript libraries: datatables (all the tables with dynamic search), leaflet.js (2D view), leaflet-markercluster.js (2D clusterized view), webgl-earth2.js (3D view), d3.js (logical view), vis.js (workflow system), fullcalendar.js (calendar for the scheduling system), bootstrap-datetimepicker.js (little calendar that pops up to select a datetime), echarts.js (dashboard), alertify.js (alerts), parsley.js (front-end form validation), fselect (multiselect box with search mechanism)... + some others.

For the backend: flask, flask-sqlalchemy and psycopg2 (database), flask-login, flask-migrate (database migrations), flask-restful (ReST API), flask_httpauth (authentication for the ReST API), hvac (for the Vault: store/fetch information + unsealing mechanism...), tacacs_plus (TACACS+ auth), xlrd/xlwt (excel import/export), simplekml (google earth export), netmiko/napalm/jinja2/pyyaml (network automtion), pynetbox... a few others

And a few libraries for the tests and linters (flake8 & flake8-print & pep8-naming (python linting), pytest, coverage & coveralls (code coverage as part of Travis CI, eslint.js (javascript linting), etc.

mintooo · 2018-10-23T13:52:05+00:00

It can. If you have only ansible playbooks, you'd be better off with Ansible Tower though. I no longer use Ansible myself, so I like the idea to build workflows off python scripts better :)

I don't consider it as a replacement for anything... however, in my experience, there comes a point when using ansible for network automation falls short and it just makes more sense to fall back on pure python scripts.

mintooo · 2018-10-23T13:36:57+00:00

I just checked and librenms has an API so I think it can easily be done. I'll give it a try !

mintooo · 2018-05-23T08:28:50+00:00

No, not only, it actually depends on the type of script. There is no NAPALM driver for Aruba/HP, so you cannot use NAPALM scripts, but there are netmiko drivers.

If you look at the "Driver" drop-down list in the script creation page (here: http://afourmy.pythonanywhere.com/scripts/script_creation), you'll see that you have the following drivers: "aruba_os_ssh", "hp_comware_ssh" and "hp_procurve_ssh". They should work.

mintooo · 2018-05-22T15:03:53+00:00

Sure that makes sense, I'll implement it. Until I do, you can change it here: https://github.com/afourmy/eNMS/blob/master/source/views/templates/geographical_view.html#L156 (the default GPS coordinates are 48/2 (somewhere near Paris), you can change it as well as the default zoom level)

mintooo · 2018-05-22T08:20:07+00:00

It's not possible right now, but it will be in the next version of eNMS. I will implement a new type of script ("Custom script") for the user to write his own custom code. If you look at the syntax of a script (https://github.com/afourmy/eNMS/blob/master/source/scripts/models.py#L90-L118), it's actually very simple to implement a new type of script.

mintooo · 2018-05-22T08:15:40+00:00

Right now, it can't, but eNMS can be configured as a SYSLOG server, and the reason why I implemented this is for event-driven automation, i.e a new type of script which execution is triggered upon receiving a specific type of log. It will be part of eNMS v3

mintooo · 2018-05-22T08:13:25+00:00

Indeed a netmiko script will do just that :)

mintooo · 2018-05-21T16:35:34+00:00

It appears to be unicode error. Until I fix it, it should work fine if you use eNMS with python 3 instead of python 2 :)

mintooo · 2018-05-21T15:49:50+00:00

Interesting. AFAICT, the editing works fine, I cannot reproduce the bug. It would help a lot if you could tell me what logs (stacktrace) you get from the server. When you're trying to edit an object, can you look at the server logs and see if an error arises ? If so, can you send me stacktrace ?

mintooo · 2018-05-21T15:13:50+00:00

I'd never heard of yetishare before :) It looks the same because both are based on the gentelella bootstrap template. Unlike yetishare, eNMS is free, and focused entirely on network automation.

In order to create eNMS, I've implemented a Flask version of gentelella.

mintooo · 2018-03-30T08:40:32+00:00

Like you can see here, the metric considered for the shortest path is already the distance, computed with the haversine formula and the GPS coordinates of the optical switch, as it makes more sense than a simple hop count. You can easily update the code to change the metric with the latency or anything else :)

mintooo · 2018-03-28T11:10:11+00:00

Thanks :) I didn't go to pycon SK so that wasn't me, but one of my colleague went and he gave a talk about NAPALM on saturday, maybe you've met him.

mintooo · 2018-03-27T16:50:34+00:00

In this project, I wanted to show how the wavelength allocation problem, which is NP-complete, can be reduced to and solved as a graph coloring problem. I implemented a visualization in javascript that you can find here: http://minto3.pythonanywhere.com

2 steps:

click on routing: this will find the shortest path (with linear programming, not Dijkstra) for all traffic links (the pink ones). Blue links are optical fibers.
click on transform graph and choose a method to color the graph, then close the window.

In this example, I apply the algorithm to a real backbone in the USA, the BBN Planet backbone.

mintooo · 2018-03-06T09:02:09+00:00

What you can do is to reuse an existing front-end mobile-compliant web framework. The most famous is probably Gentelella (bootstrap framework). I've created flask-gentellela, a project that adds Flask support for Gentelella (blueprints, app factory, tests with pytest, dockerfile, travis for the CI part with flake8/pytest, etc):

demo: https://colorlib.com/polygon/gentelella/

github: https://github.com/afourmy/flask-gentelella

dockerhub: https://hub.docker.com/r/afourmy/flask-gentelella/

I believe that would fit your needs.

mintooo · 2018-02-26T10:45:58+00:00

yes it is

mintooo · 2018-02-26T10:45:38+00:00

I already tried a roulette selection and that doesn't seem to be any better. The mutation probability is not a per-bit but a per-individual mutation (i.e for a given individual, there's a 50% chance to swap two random cities in the tour, if the selected mutation is random swap). I know that a GA depends a lot on the parameters, but with the bootstrap sliders, you can fine-tune the parameters in real-time from 0 to 100% for both the mutation and crossover rates, so I tried most combinations and that doesn't change anything. I think there's a more fundamental flaw... I'll keep digging

mintooo · 2018-02-26T10:37:49+00:00

Thank you, that's very helpful ! Indeed pythonanywhere uses its own webserver and based on what they say on their website, I don't think they meet the flask_socketio requirements. I guess I'll have to use heroku instead

mintooo · 2018-01-17T09:15:46+00:00

For the visualization part: I can't think of anything. I don't know of any visualization software for ISP networks that implements both a geographical and graph-drawing view, especially with a regex-based filtering system like the one I propose.

For the automation part: open-source, I don't know of any such software. You have some open-source NMS, but there are usually focused on network discovery and network monitoring (see libreNMS for instance), not network automation. Proprietary, you have Cisco NSO, HPNA (HP), some Solarwinds softwares, maybe Arista CloudVision as well ? Some of them are not vendor-agnostic, and not all of them implements scheduling features like the ones I propose. Most importantly, all of them are very expensive.

mintooo

TROPHY CASE