This is just a small post on how to enable SNMP on a Lastline Advanced Malware Protection appliance in order to query the basic host and network MIBs from an SNMP monitoring server. Note that this is not the preferred method of monitoring a Lastline device. The Product API (PAPI) should be used instead such as shown in the online docs. However, basic SNMP gives access to the CPU, memory, load average and the network interface statistics incl. the anonymous VPN tunnel interface.

Since all Lastline devices are basically a Ubuntu server, the basic setup for SNMP is quite similar to my tutorial for a generic Linux. The only step missing there is the allow statement for the Uncomplicated Firewall (ufw).

The basic steps are the following. Install the snmpd package and verify that it is running:

sudo apt-get update
sudo apt-get install snmpd snmp

monitoring@jw-lastline01:~$ sudo netstat -tulpen | grep snmp
udp        0      0 0.0.0.0:39103           0.0.0.0:*                           0          229940943   3833/snmpd
udp        0      0 127.0.0.1:161           0.0.0.0:*                           0          229940944   3833/snmpd

Note that the snmpd is only listening on the localhost (127.0.0.1) IPv4 address so far. Change the following settings within the conf file in order to listen on any IPv4 address and to be able to read out everything with the configured community string:

sudo nano /etc/snmp/snmpd.conf

agentAddress udp:161
rocommunity THISISTHEKEY 192.168.0.0/16

sudo service snmpd restart

monitoring@jw-lastline01:~$ sudo netstat -tulpen | grep snmp
udp        0      0 0.0.0.0:161             0.0.0.0:*                           0          230009702   22622/snmpd
udp        0      0 0.0.0.0:33882           0.0.0.0:*                           0          230009701   22622/snmpd

Now, the daemon is listening on every local IPv4 address (0.0.0.0).

The Lastline Ubuntu image uses an uncomplicated firewall (ufw) whose policy looks like that:

monitoring@jw-lastline01:~$ sudo ufw status
Status: active

To                         Action      From
--                         ------      ----
10025/tcp                  ALLOW       Anywhere
123/udp                    ALLOW       Anywhere
22/tcp                     ALLOW       Anywhere
80/tcp                     ALLOW       Anywhere
1194/udp                   ALLOW       Anywhere
443/tcp                    ALLOW       Anywhere
Anywhere                   ALLOW       198.19.0.0/16

That is: SNMP is not yet permitted. To allow it, simply add the following rule via:

monitoring@jw-lastline01:~$ sudo ufw allow snmp
Rule added

monitoring@jw-lastline01:~$ sudo ufw status
Status: active

To                         Action      From
--                         ------      ----
10025/tcp                  ALLOW       Anywhere
123/udp                    ALLOW       Anywhere
22/tcp                     ALLOW       Anywhere
80/tcp                     ALLOW       Anywhere
1194/udp                   ALLOW       Anywhere
443/tcp                    ALLOW       Anywhere
Anywhere                   ALLOW       198.19.0.0/16
161                        ALLOW       Anywhere

A basic try with snmpwalk on the machine itself looks like this:

monitoring@jw-lastline01:~$ snmpwalk -v 2c -c THISISTHEKEY 192.168.120.20 .1.3.6
iso.3.6.1.2.1.1.1.0 = STRING: "Linux jw-lastline01 3.13.0-100-generic #147~precise1-Ubuntu SMP Wed Oct 19 00:06:40 UTC 2016 x86_64"
iso.3.6.1.2.1.1.2.0 = OID: iso.3.6.1.4.1.8072.3.2.10
iso.3.6.1.2.1.1.3.0 = Timeticks: (979) 0:00:09.79
iso.3.6.1.2.1.1.4.0 = STRING: "Me <me@example.org>"
iso.3.6.1.2.1.1.5.0 = STRING: "jw-lastline01"
iso.3.6.1.2.1.1.6.0 = STRING: "Sitting on the Dock of the Bay"
iso.3.6.1.2.1.1.7.0 = INTEGER: 72
^C

Now use your SNMP monitoring software such as I am using MRTG with Routers2 (old but stable) such as shown here. To my mind, at least the interface statistics for the anonymous VPN tunnels (llanonvpn0 and llanonvpn1) are interesting because you don’t see them in the GUI under Appliance -> Metrics. Here are some sample graphs from my Lastline pinbox in the monthly view:

Cheers.

Featured image: “Dell” by Craig Rodway is licensed under CC BY-NC-ND 2.0.

If you are using a Lastline device (Manager, Engine, Sensor or Pinbox) you can reach the machine via SSH after you activated it via

monitoring_user_password

This is a small record on how to add a public key to the Lastline device. However, it is quite general since the Lastline appliance is built upon a standard Ubuntu server.

Just as a quick reminder, this is how you enable the SSH login on a Lastline box in general. Note that this requires console access:

lastline_setup
monitoring_user_password <password>
save

(The following guide is not an official guide from Lastline! Talk to the Lastline support or to your SE in order to confirm this for your installation.)

Adding Public Keys

The main goal is to copy the public ssh-key into the 

~/.ssh/authorized_keys

There are many ways to copy the public key into the authorized_keys file. If you are on the local machine you can concatenate it with “>>” such as:

cat id_rsa.pub >> authorized_keys

Another way is to use

ssh-copy-id

weberjoh@jw-nb15-lx:~$ ssh-copy-id -f monitoring@192.168.120.20
/usr/bin/ssh-copy-id: INFO: Source of key(s) to be installed: "/home/weberjoh/.ssh/id_rsa.pub"
The authenticity of host '192.168.120.20 (192.168.120.20)' can't be established.
RSA key fingerprint is SHA256:JqNJhtSBgn0AU9Ka8hZHBxHhjFfD3+RpvpcA81jwwN8.
No matching host key fingerprint found in DNS.
Are you sure you want to continue connecting (yes/no)? yes
monitoring@192.168.120.20's password:

Number of key(s) added: 1

Now try logging into the machine, with:   "ssh 'monitoring@192.168.120.20'"
and check to make sure that only the key(s) you wanted were added.

In any case you should check that the file permissions are rw only for the user! If not, change it with:

sudo chmod 0600 ~/.ssh/*

The following listing shows the ~/.ssh/ folder on my Lastline Pinbox. Note the authorized_keys file with the correct permissions and its content:

monitoring@jw-lastline01:~/.ssh$ ls -ahl
total 12K
drwxr-x--- 2 monitoring monitoring 4.0K Jan 23 15:32 .
drwxr-xr-x 5 monitoring monitoring 4.0K Jan 19 13:48 ..
-rw------- 1 monitoring monitoring  404 Jan 23 15:32 authorized_keys
monitoring@jw-lastline01:~/.ssh$ cat authorized_keys
ssh-rsa AAAAAAAfoobarAAAAABBBBBandevenmuchmorelongerCCCCCCC johannes@webernetz.net

That’s it! You are now able to log in with the SSH key-based authentication. For example with PuTTY this looks like that. Note the third line which requires to type in the passphrase for the private key. Also note that there was no challenge for the password itself:

login as: monitoring
Authenticating with public key "imported-openssh-key"
Passphrase for key "imported-openssh-key":
Welcome to Ubuntu 12.04.5 LTS (GNU/Linux 3.13.0-100-generic x86_64)

 * Documentation: https://update.lastline.com/updates/distros/Lastline_Enterprise_Pinbox_Installation_Manual.pdf .

 * To test the status of this Lastline appliance, please execute "lastline_test_appliance".

Last login: Mon Jan 23 15:33:21 2017 from 192.168.7.53

[Disabling Password Authentication]

Though it is possible to disable the password authentication at all it is not officially acknowledged by Lastline. However, if you are willing to test it configure the following within the sshd_config file. (Note: NOT the ssh_config but the sshd_config. Note the “d” for daemon that is listening to incoming ssh connections.):

sudo nano /etc/ssh/sshd_config

PasswordAuthentication no

sudo service ssh restart

Ciao.

Featured image: “Data Center” by Bob Mical is licensed under CC BY-NC 2.0.

While preparing for my CCNP SWITCH exam I built a laboratory with 4 switches, 3 routers and 2 workstations in order to test almost all layer 2/3 protocols that are related to network management traffic. And because “PCAP or it didn’t happen” I captured 22 of these protocols to further investigate them with Wireshark. Oh oh, I remember the good old times where I merely used unmanaged layer 2 switches.

In this blogpost I am publishing the captured pcap file with all of these 22 protocols. I am further listing 45 CHALLENGES as an exercise for the reader. Feel free to download the pcap and to test your protocol skills with Wireshark! Use the comment section below for posting your answers.

Of course I am running my lab fully dual-stacked, i.e., with IPv6 and legacy IP. On some switches the SDM template must be changed to be IPv6 capable such as

sdm prefer dual-ipv4-and-ipv6 default

(In two upcoming posts I will first list many show commands from the routers/switches in the lab that are related to the CCNP SWITCH protocols, while in a third one I will show all answers to the proposed questions here with appropriate Wireshark screenshots. And if you are interested in some more pcap files on my site follow the pcap tag. Posts with some Wireshark screenshots and explanations are tagged with Wireshark.)

The pcap file consists of 3893 packets (with 50 % spanning-tree messages). The following zip file is 72 kb in size with this sha256 hash: 4274C8C61BCFB3ADE0557535164AFCEB7FCE47C59601DBFB85059909D5FF8F32.

The challenges are basically related to the protocols but a few can only be answered when using Wireshark and some of its features. If you want to answer all challenges you can use the following “template” for them. In Wireshark you can often use the right-mouse-click with Copy ->  Value. Please use the comment section to post your answers!

STP1:
STP2:
STP3:
CDP1:
CDP2:
VTP1:
VTP2:
VTP3:
DTP:
LACP1:
LACP2:
LLDP:
UDLD:
HSRP:
DHCP1:
DHCP2:
DHCP3:
DHCP4:
SNMP:
Syslog:
NTP:
IP SLA:
DNS1:
DNS2:
DNS3:
DNS4:
TFTP1:
TFTP2:
TFTP3:
TFTP4:
ARP:
LOOP:
RIP:
RIPng:
ICMPv6 RS/RA1:
ICMPv6 RS/RA2:
ICMPv6 NS/NA:
ICMPv6 Ping:
ICMP:
SSH:
BONUS1:
BONUS2:
BONUS3:
BONUS4:
BONUS5:

Protocols, Protocols, Protocols

Of course I preferred IPv6 rather than legacy IP for all “above layer 2” protocols. Examples are DNS, NTP, SNMP, and SSH. In the following I am only listing the protocols by name and their basic configuration. If you are not familiar with any of these protocols, please wikipedia it yourself. Here we go:

• STP (Spanning Tree Protocol) <- set to

  spanning-tree mode rapid-pvst

  , appears per default every 2 seconds on all switch ports. I additionally use 

  spanning-tree loopguard default

   

  spanning-tree portfast default

   , and

  spanning-tree portfast bpduguard default

   . CHALLENGE: What is the root bridge identifier for VLAN 121? 2nd CHALLENGE: At what time (relative to the beginning of the trace) did the first topology change occur? 3rd CHALLENGE (a bit tricky): What is the port identifier in VLAN 30 from bridge “00:21:1b:ae:31:80”. I mean, what is the ID when it is correctly split into the port prio and the port number? That is, how is the ID displayed on a Cisco device when the “show spanning-tree” command is issued? (Hint: This is the 1998 version of 802.1D, not the 2004.)
• CDP (Cisco Discovery Protocol) <- appears per default every few seconds. CHALLENGE: What is the Cisco IOS Software version running on the CCNP-LAB-S1 switch? 2nd CHALLENGE: What is the native VLAN from the same device?
• VTP (VLAN Trunking Protocol) <- manually added a new VLAN on the VTP server while capturing. CHALLENGE: What is the management domain of VTP? 2nd CHALLENGE: What is the name von VLAN 30? (Hint: The VLAN ID inside the “ISL VLAN ID” field is not in decimal but hexadecimal notation.) 3rd CHALLENGE: What is the last configuration revision?
• DTP (Dynamic Trunking Protocol) <- appears per default. CHALLENGE: What is the “Trunk Operating Status” from the sender ID “00:21:1b:ae:31:99”?
• LACP (Link Aggregation Control Protocol) <- appears on a port that is within a channel-group active|passive. CHALLENGE: What is the “Actor” and “Partner” port as seen from the actor “00:0a:8a:a1:5a:80”? 2nd CHALLENGE: What is the partner system priority?
• LLDP (Link Layer Discovery Protocol) <- after

  lldp run

    on global configuration. CHALLENGE: What is the IPv6 management address of CCNP-LAB-S1?
• UDLD (Unidirectional Link Detection; where is the P for Protocol here?!?) <- activated globally with

  udld aggressive

   , per default only on fiber ports. CHALLENGE: Which fiber interface port on device “FOC0630Z3KZ” is used?
• HSRP (Hot Standby Routing Protocol) <- Version 2 for IPv6 and legacy IP, both with md5 authentication. CHALLENGE: What is the virtual IPv6 address for HSRP group 127?
• DHCP (Dynamic Host Configuration Protocol) <- since I have configured several VLANs with DHCP server, CHALLENGE: What is the client MAC address that requests an IPv4 address? 2nd CHALLENGE: What is the hosts vendor? 3rd CHALLENGE: Which IPv4 address did it request? 4th CHALLENGE: What IPv4 address did it actual get?
• SNMPv2c (Simple Network Management Protocol) <- I am using my MRTG/Routers2 for querying the three routers in this lab. Yes, you can read out my SNMP community string in plaintext. CHALLENGE: What is the community string?
• Syslog <- though not covered explicitly in CCNP SWITCH. CHALLENGE: What kind of information was sent by 192.168.121.2?
• NTP (Network Time Protocol) <- periodically, configured with IPv6. CHALLENGE: What type of NTP server is the configured IPv6 server? I mean: Which type of clock does it use?
• IP SLA (Internet Protocol Service Level Agreement) <-

  ip sla responder

    on one router, “ip sla … udp-jitter …” on another. UDP Port 1967, it seems that Wireshark has no decoder for this Cisco proprietary protocol, display filter:

  udp.dstport == 1967 || udp.srcport == 1967

  . CHALLENGE: Which two routers (IPv4 address) are the SLA responders?
• DNS (Domain Name System) <- manually when pinging. CHALLENGE: What is the IPv6 address of the DNS server? 2nd CHALLENGE: Which name was queried? 3rd CHALLENGE: How many hops is the DNS server away? 4th CHALLENGE: Is the response DNSSEC validated?
• TFTP (Trivial File Transfer Protocol) <- automatically after every “copy run start” due to my “archive” configuration on the routers. Yes, you can read out my complete router config in the tcpdump trace. Feel free to crack my trivial passwords. (You can also use the tool NetworkMiner to extract the file.) CHALLENGE: What is the password of the user “weberjoh”? 2nd CHALLENGE: What is the “enable secret” password? 3rd & 4th CHALLENGE: What are the passwords in plaintext? Tip: hashcat.
• ARP (Address Resolution Protocol) <- Normal IPv4 networking stuff. CHALLENGE: “192.168.121.1 is at ???”
• LOOP (Ethernet Configuration Testing Protocol) <- What is this? Have never heard of it so far. CHALLENGE: How many different MAC addresses sourced a loop frame?

The following protocols are not directly mentioned within the CCNP SWITCH course but they appear in the trace since I used them for some reasons:

• RIPv2 (Routing Information Protocol) <- between my two routers, because it is simple and I am always using OSPF anywhere, so I wanted to test RIP in here. (Refer to this blogpost for OSPFv2 for IPv4.) CHALLENGE: Which two routers (IPv4 source addresses) advertise the default route with a metric of 1?
• RIPng <- same as above. Yes, I know, you are seeing my global unicast prefixes here. But I don’t care – it’s only a lab. And my /48 prefix is easy to find out anyway. I tuned the timers a little bit to have a faster convergence time:

  timers 10 30 10 20

   . (If you are interested in OSPFv3 for IPv6 refer to this blogpost.) CHALLENGE: Which two routers (IPv6 link-local address) advertise the default route?
• ICMPv6 RS/RA (Internet Control Message Protocol) Router Solicitation/Advertisement,

  icmpv6.type == 133 || icmpv6.type == 134

    <- because I have configured at least one VLAN with IPv6. CHALLENGE: Which IPv6 prefixes are advertised? 2nd CHALLENGE: How many router solicitations are directly answered with router advertisements?
• ICMPv6 NS/NA (Internet Control Message Protocol) Neighbor Solicitation/Advertisement,

  icmpv6.type == 135 || icmpv6.type == 136

    <- Normal IPv6 traffic. CHALLENGE: Which global unicast IPv6 address is requested in the duplicate address detection (DAD) process?
• ICMPv6 Ping,

  icmpv6.type == 128 || icmpv6.type == 129

    <- Every 60 seconds from the “IP SLA” configuration. And five pings after a manual ping from the router CLI. CHALLENGE: What is the value of the “Data” field for the ping to “2a01:488:42:1000:50ed:8588:8a:c570”?
• ICMP for legacy IP <- I have no classical pings in my trace but timestamp request/reply packets due to the “icmp-jitter” test within an IP SLA. CHALLENGE: How many request/reply packages are sent during each run of the IP SLA?
• SSH (Secure Shell) via IPv6 to ccnp-lab-r2, fun fact: This is the only

  tcp.stream

    within the whole trace. CHALLENGE: What is the client protocol version in the SSH session?

BONUS CHALLENGE: There is still one protocol left in the trace. Which one?
2nd BONUS CHALLENGE: Furthermore, there are some UDP connections from/with high ports on both sides. What are these connections related to?
3rd BONUS CHALLENGE: How many conversations over IPv6/IPv4/TCP/UDP are in the trace?
4th BONUS CHALLENGE: What is the average “packets per second” through this trace file?
5th BONUS CHALLENGE: How many IPv6/IPv4 packets are in the trace?

But still not all

I am sorry, but the following protocols/technologies are NOT inside in this pcap trace though they are covered in the SWITCH course:

• PAgP (Port Aggregation Protocol, Cisco proprietary) <- since I am using LACP.
• MST (Multiple Spanning Tree Protocol) <- since I am using PVST+.
• RADIUS (Remote Authentication Dial-In User Service)
• TACACS+ (Terminal Access Controller Access-Control System Plus)
• EAPOL (Extensible Authentication Protocol over LAN)
• SNTP (Simple Network Time Protocol) <- but NTP.
• SNMPv3 <- sorry about that. Lack of time. I used SNMPv2c.
• VRRP (Virtual Router Redundancy Protocol) <- because I have HSRP already.
• GLBP (Gateway Load Balancing Protocol) <- dito, you can’t have everything.

What I have missed (it’s too late to do the trace for the hundredth time):

• Telnet <- should not be used anyway
• ICMP for legacy IP with ping (simple echo-request)
• Traceroute <- that is, decreasing TTLs

My Lab

Though not really interesting in respect of the protocols, this is the CCNP SWITCH lab I used to create the pcap trace. Note the green “capture” arrow which shows on which interface (gi0/1 on S1) I took it:

It consits of the following components:

• 1x Cisco Catalyst 2960, (C2960-LANBASEK9-M), Version 15.0(2)SE9
• 2x Cisco Catalyst 2950, (C2950-I6K2L2Q4-M), Version 12.1(22)EA14
• 1x Cisco Catalast 3560, (C3560-IPSERVICESK9-M), Version 12.2(55)SE10
• 3x Cisco Router 2811, (C2800NM-ADVENTERPRISEK9-M), Version 15.1(4)M9
• 2x old Notebooks, Dell or somewhat, running either Ubuntu or Knoppix Linux

I am using

switchport trunk native vlan 2

switchport access vlan 3

I captured on switch S1 with the following commands:

monitor session 1 source interface Gi0/1
monitor session 1 destination interface Fa0/18 encapsulation replicate

CCNP-LAB-S1#show monitor detail
Session 1
---------
Type                   : Local Session
Description            : -
Source Ports           :
    RX Only            : None
    TX Only            : None
    Both               : Gi0/1
Source VLANs           :
    RX Only            : None
    TX Only            : None
    Both               : None
Source RSPAN VLAN      : None
Destination Ports      : Fa0/18
    Encapsulation      : Replicate
          Ingress      : Disabled
Filter VLANs           : None
Dest RSPAN VLAN        : None

I am still a bit confused about the “encapsulation replicate”. Without it no layer 2 traffic was monitored. However, now I have the problem that many but not all packets in the pcap file have the 802.1Q vlan tag. This irritates when scrolling through the packets.

I used the following sequence during the 5 minute trace:

1. Topology change:

   shutdown

     of gi0/2 on switch S1.
2. Creation of new VLAN on STP server switch S1.
3. Reboot of router R1b for HSRP and routing changes.

4. shutdown

     and

   no shut

     of access port eth0/6 on switch S2 with Knoppix Linux machine for DHCP/ICMPv6.
5. SSH to router R2 for at least one tcp.stream.

6. copy run start

     on router R2 to trigger TFTP copy.

7. clear hosts

     and

   ping ip.webernetz.net

     from router R2 for DNS and ICMPv6.
8. Another topology change:

   no shut

     of gi0/2 on switch S1.

Have fun. And happy easter. The Lord is risen indeed!

Featured image: “Data Center” by Bob Mical is licensed under CC BY-NC 2.0.