Thursday, 28 October 2021

DevNet - Linux - Bash (1)

 BASH basics

- acronym for "Bourne Again Shell"

- allows for command and script processing

- supports piping i.e.:

$env | more 

will add page breaks when displaying the information:




- For help  "man" can be used to access detailed information about command i.e.:

$man pwd

will return:



- sample commands:

pwd - print current working directory

ls - list files and directories, optionally can be used with "-a" (show all including hidden files) or "-l" to list permissions and user/group ownership, example:


mkdir - create directory

rm - deletes file or directory, useful flag "-rf" to force remove all files within deleted

        folder

cp - copy file or folder, does not delete the source file

mv - move file/folder between directories, also can be used to rename files/folders

        using "mv -f " will force overwrite if destination file already exists

cat - can be used to view or create files, 

        useful example $cat filename.txt | more 

touch - used to create an empty file or change the time stamp without opening it

- running commands at the admin level "sudo" can be used, example:

  $sudo apt-get update - this will prompt user to enter password before proceeding with an update of the list of available packages

- environment variables:

env - to view current set of variables, example: $env | more

echo - can be used to display single variable i.e.: echo $PATH

export - can be used to add new variable i.e.: export PASS=c1sco

unset - can be used to remove variable i.e.: unset PASS


note! newly created variable will be lost after session reload unless it is added to .bashrc file (or .zshrc on MacOS) 

to add variable, example:

$echo "export PASS=c1sco" >> .bashrc

to reload the variables:

$source ~/.bashrc or $. ~/.bashrc 

Sunday, 17 October 2021

DevNet - Python module Urllib3 - user-friendly HTTP client

 Urllib3 is a HTTP client for python it can grab data, post data, stream data, work with JSON as well as use redirects.

Urllib3 can be installed using pip:

    python -m pip install urllib3

    or via GitHub:

    git clone git://github.com/urllib3/urllib3.git

    python setup.py install


code example:

 cat test_urllib 

#!/usr/bin/env python3


"""urllib3 module test"""


import urllib3


http = urllib3.PoolManager()

URL = 'http://dub-ne.blogspot.com'

resp = http.request('GET', URL)

print(resp.status)


- this example create GET request to www.dub-ne.blogspot.com and prints the return code of response, output:
 

python test_urllib 

200

- code 200 ("OK") indicates that the request has succeeded.


HTTP response code groups:

  • Informational responses (100–199)
  • Successful responses (200–299)
  • Redirects (300–399)
  • Client errors (400–499)
  • Server errors (500–599)


Urllib3 features:

  • Thread safety.
  • Connection pooling.
  • Client-side SSL/TLS verification.
  • File uploads with multipart encoding.
  • Helpers for retrying requests and dealing with HTTP redirects.
  • Support for gzip, deflate, and brotli encoding.
  • Proxy support for HTTP and SOCKS.
  • 100% test coverage.

Documentation: urllib3.readthedocs.io

DevNet - Parsing data formats in Pyton: JSON & YAML (basics)

JSON data code example.

- JSON file:



{ "node": 

  { "hostname": "Edge1", "details": { "interface":

   { "name": "Ethernet0", "description": "L2 Uplink", "enabled": "true"

   }

                                    }

  }

  }


- Python file:


#!/usr/bin/env python

""" simple code to parse json """

import json

with open("data.json", "r", encoding = "UTF-8") as file:

    data = json.load(file)

    print(data)


- Output: 


python parse_json.py

{'node': {'hostname': 'Edge1', 'details': {'interface': {'name': 'Ethernet0', 'description': 'L2 Uplink', 'enabled': ‘true'}}}}


*** of course we can narrow the output information by modifying print command to for instance:


print(data['node']['hostname'])


which would return only the hostname:

python parse_json.py

Edge1


*** there are four ways to be used when parsing data:
    - load() - import native JSON as Python dictionary 
    - loads() - import JSON data from the string
    - dump() - used to write JSON data from python objects into JSON file
    - dumps() - same as dump() except returns string, does not require file object

YAML example:

- YAML file:

cat data.yaml  

---

device:

    hostname: Edge1

    interface:

        name:Ethernet0

        description:Uplink

        enabled:true

...


- Python file:


cat parse_yaml.py 

""" parse YAML file simple example """

import yaml


with open ("data.yaml", "r", encoding = "UTF-8") as fdata:

    ydata = yaml.safe_load(fdata)

    print(ydata)



- Output


python parse_yaml.py

{'device': {'hostname': 'Edge1', 'interface': 'name:Ethernet0 description:Uplink enabled:true’}}


*** two functions are available with in YAML module:

   - yaml.load - to convert YAML data into Python
   - yaml.dump - to convert Python data back to YAML

-----

All files are available on my GitHub: https://github.com/lightarchivist/DevNetLabs

Sunday, 3 October 2021

DevNet - XML - Parsing Data Formats in Python

 We should think of data formats in terms of different ways of providing structure and consistency to present the data.


->XML - Python natively supports XML encoding and decoding. Parsing XML data is not straight forward way when compared to YAML and JSON.


The simplest way is to parse the XML file is to convert it to ordered dictionary using xmltodict module. Ordered dictionary is a subclass of dictionary in Python and it is used because it remembers the order of how the keys are inserted in to dictionary.


Example:


XML file:


<?xml version="1.0"?>

<device>

 <hostname>Edge1</hostname>

 <interface>

  <name>Ethernet0</name>

  <description>L2 Uplink</description>

  <enabled>true</enabled>

 </interface>

</device>



Install xmltodict module: pip install xmltodict


Python file:


import xmltodict

xmlFile = open ('lab.xml')

xmlData = xmlFile.read()

xmlFile.close()

xml_dict = xmltodict.parse(xmlData)

print(xml_dict)



Output after running Python script:


OrderedDict([('device', OrderedDict([('hostname', 'Edge1'), ('interface', OrderedDict([('name', 'Ethernet0'), ('description', 'L2 Uplink'), ('enabled', 'true')]))]))])



Additional info: link to tutorial how to use xml.etree.ElementTree module:


https://docs.python.org/3.8/library/xml.etree.elementtree.html



All examples are available on my GitHub: https://github.com/lightarchivist/DevNetLabs

Plumbing... QoS

Rule no 1. QoS does not help in situations where there is no enough bandwidth but helps optimize performance by prioritization of the traffi...