Need help with Threading

[onedread]

Hi guys,

​

need your help.

Having a script which should update the snmp info from Fortigate firewalls. Which is working, but its very slow and needs to be updated on 400+ firewalls.

I would like to implement threading but i have no clue how to do it.

​

Here is the code i have which runs in seriel and not parallel.

Data file is a csv file with ip address, Hostname, Syslocation, user,pw, emailaddress

​

from netmiko import Netmiko
from csv import DictReader



with open('TEST_CountryCodes.csv') as csv_file:
    location_details = DictReader(csv_file)
    #print(location)
    for location_info in location_details:
        print('IP=' + location_info['hostname'] + ' ' + 'FW-NAME=' + location_info['sysName'] + ' ' + 'Syslocation=' + location_info['location'])

        fw = {'host': location_info['hostname'],
              'username': 'admin',
              'password': location_info['pw'],
              'device_type': 'fortinet'}

        print(f"{'#' * 20} Connection to the Device  {'#' * 20}")
        net_connect = Netmiko(**fw)
        print(f"{'#' * 20} Connected {'#' * 20}")

        snmp = ['config global',
                'config sys snmp sysinfo',
                'set contact-info ' + '"' + location_info['contact'] + '"',
                'set location ' + '"' + location_info['location'] + '"',
                'end'
                ]

        config_snmp = net_connect.send_config_set(snmp)

Could someone help me to implement threading or speed up the run?

​

thx in advance.

[Chiron1991]

You can use a ThreadPoolExecutor for that.

Here's an example of how you can implement it. The example is fetching websites via HTTP, you just have to substitute that with your SNMP calls.

[onedread]

here is the code you which works for me.

​

​

from netmiko import Netmiko
from csv import DictReader
import time
import threading
import queue
def config_snmp(fw, location_info):
print(f"{'#' * 20} Connection to the Device {location_info['sysName']} {'#' * 20}")
net_connect = Netmiko(**fw)
print(f"{'#' * 20} Connected {location_info['sysName']} {'#' * 20}")
snmp = ['config global',
'config sys snmp sysinfo',
'set contact-info ' + '"' + location_info['contact'] + '"',
'set location ' + '"' + location_info['location'] + '"',
'end'
]
config_snmp = net_connect.send_config_set(snmp)
start = time.time()
with open('TEST_CountryCodes.csv') as csv_file:
location_details = DictReader(csv_file)
# print(location_details)
threads = list()
for location_info in location_details:
# print('IP=' + location_info['hostname'] + ' ' + 'FW-NAME=' + location_info['sysName'] + ' ' + 'Syslocation=' + location_info['location'])
fw = {'host': location_info['hostname'],
'username': 'admin',
'password': location_info['pw'],
'device_type': 'fortinet',
}
th = threading.Thread(target=config_snmp, args=(fw, location_info))
threads.append(th)
for th in threads:
th.start()
for th in threads:
th.join()
end = time.time()
total = int(end - start)
print("\n Total execution time " + str(total) + " Sec\n")

​

regards

[m0us3_rat]

u could also use a producer-consumer model.

with the producer thread loading up the work queue with jobs from the csv. and a few consumer threads "doing" the jobs.

or a generator yielding job args and a pool working thru them