The computer revolution continues to move on and present-day networks continue to expand into access networks and access protocols never originally envisioned when Ethernet was first introduced.
Ethernet network designers and operators as well as equipment vendors are facing many of the same decisions that once concerned enterprise and carrier networks. Do we base a network on Layer 2 or Layer 3 switching? This issue was settled many years ago, for Enterprise and Carrier networks. The ITS and SCADA community is evolving as network threats increase. Industrial and machine networks are still on a learning curve, as their relative importance is on the rise.
ITS, Industrial and SCADA networks for the most part are insecure and open to attack. Their vulnerabilities are due mostly to the following:
- Flat layer 2 design for all things connected.
- The use of too big a subnet.
- The lack of the use of VLAN’s (Virtual Local Area Networks) for segmentation.
- Network Operations too lazy to develop an IP addressing scheme.
- The use of non-secure wireless transport and access points.
- Failure to Encrypt
- Failure to use Device Binding
- Elimination of promiscuous binding
- Simple passwords, no passwords, or compromised passwords.
- Non-existent network and device monitoring.
- Failure to turn off unused ports.
- Failure to put unused ports into a limited vlan.
- Failure to lock access ports with Sticky MAC/IP.
- Reliance on “FIREWALL” for security.
- Failure to do yearly Firewall audit
- Failure to remove unused ports and policies
- Failure to understand how the firewall works
- Lack of trained personnel.
- Lack of or incomplete network documentation.
- I have it all memorized
- If we write it down it may be compromised.
- Too hard to update because of changes.
- We do not want to change the way we work.
- Never heard of ITIL – or process flow control
- Connectivity to the Enterprise network (This is incredibly stupid)
- Failure to understand the difference between Enterprise and Industrial Networks
- Enterprise networks go down ALL the time – maintenance, updates, & failures.
- Industrial Networks are EXPECTED to be up all the time. 24 x 7 x 365
- Enterprise networking equipment is cheaply built
- Requires cooling
- Equipment has fans that introduce mechanical failure
- Requires special clean room (Data Center)
- Expensive maintenance contracts
- Industrial networking equipment is ruggedly built – usually no fans
- Uses less electricity – lower operation costs
- Wider temperature range for operations (-40C ~ +85C
- Ruggedized for outdoors and harsh industrial environments
- 5 Year or Limited Lifetime Warranty
- Vendors idiotically renaming unmanaged devices – “Self-Managed”
- Managers and Administrators do not know the difference
- It’s not the vendor who will suffer the outage – it is you
- If you can access critical infrastructure from the Reception Desk – you have a problem.
- You have a reverse problem if you network is run by a data dictator who turns off network services such as telnet / lldp in the name of security.
Layer 2 switching offers the illusion of easy setup, operation, and maintenance. However, just as it was the case for enterprise and carrier networks, Layer 2 is not adequate to the task of securing the network. It is a sad fact that many Layer 2 networks have security features that go unused or left in a compromised setting (Think – default passwords / simple passwords) It is a fact that Layer 3 architectures are easier to manage, provide better performance and allow significantly greater scalability. The bottom line is that Layer 3 switching creates the optimum architecture for access networks.
Think about the differences in the network layers as we take a closer look at the control differences between the DATA LAYER (Layer 2) and the NETWORK LAYER (Layer 3), as defined by the “The Open System Interconnection (OSI) model” that defines a networking framework to implement protocols in seven layers.
LAYER 2 AND 3 DIFFERENCES
Enterprise and commercial networks (ITS, SCADA, Control), including access networks, must be managed networks, whatever their underlying network architecture. Although Layer 2 and 3 networks require the same amount of Internet Protocol (IP) infrastructure and planning, only Layer 3 architectures can take advantage of automated tools to ease deployment; only Layer 3 architectures provide nearly unlimited flexibility in mixing media and network technologies; and only Layer 3 routing-switching provides built-in isolation between network segments to make troubleshooting and fault management much simpler.
Layer 2 proponents claim that its architecture is superior because they don’t require planning for and establishing an IP infrastructure, including addressing assignments and sub network boundaries. Network segmentation requires the use of VLAN’s at Layer 2, and inter-vlan communication requires Layer 3 routing. It is a fact is that any network that delivers IP services — Internet access, voice over IP, IP video, etc. — requires some sort of an IP infrastructure. The fact that Layer 3 architectures require operators to plan and engineer their networks, those architectures ultimately result in better performance and superior network services.
Once network deployments move beyond the planning stage and into actual operation, the benefits of Layer 3 architectures become very clear. Layer 3 technology based on IP is very mature and widely deployed by enterprises and carriers. With such a large market to serve, engineers have developed powerful and convenient tools to automate and simplify network configuration and operation. These tools that are lacking in Layer 2 only environments.
As an example: consider how the two architectures distinguish between network elements. In a Layer 3 network, the primary identifier is an IP address. Network operators and operations have complete flexibility in assigning these IP addresses to the various systems, so they can assign them in a manner that most appropriately matches their engineered deployment strategy. There are also tools such as the Domain Name System (DNS) that easily map IP addresses into human-readable names. In addition, there are several tools such as the Dynamic Host Configuration Protocol that can assign IP addresses and other IP configuration information to network elements automatically, without manual intervention.
Contrast the flexibility and automation available with IP addressing to that of the typical Layer 2 system identifier: The Media Access Control (MAC) address. Equipment manufacturers, not network operators, assign MAC addresses, leaving operators no control over the mapping. Unlike an IP address, a device’s MAC address often changes when the device is repaired or replaced, significantly increasing the burden on the operator’s to actively maintaining the network. There is no mechanism like DNS to map MAC addresses to names, so operators are forced to use cryptic, 12-digit hexadecimal values to identify systems. Very few system engineers can predict failover and automatic occurrences due to the complexity of spanning tree – CIST / MSP operations.
Engineers also encounter problems in network operations attempting to logically segment their Layer 2 networks. While Layer 3 networks are naturally segmented by IP sub networks without any special work on the operator’s part, the Layer 2 equivalent, a virtual LAN (VLAN), must be manually configured in each device.
Another significant advantage of Layer 3 architecture is their inherent support for multiple network technologies. Layer 2 networks are confined to a Layer 2 technology such as Ethernet or Asynchronous Transfer Mode. Layer 3 architectures, on the other hand, can encompass all network technologies, protocols and still provide a common core management and operational infrastructure.
Once a network is operational, Layer 3 architectures greatly simplify troubleshooting and fault management. For example, common troubleshooting tools automatically use reverse DNS to display captured network traffic using human-readable names. In contrast, a technician diagnosing Layer 2 architectures is limited to looking at 12-character, hexadecimal MAC addresses. Trace Route is an invaluable Layer 3 tools to map out the path of an ip packet.
Troubleshooting is also easier when problems are easier to isolate. In Layer 3 architecture, we have many connected sub networks in which isolation is inherent in their segmentation. (Think VLAN for DATA / VLAN for Video, etc.) Routers that interconnect sub networks ensure that only appropriate, Layer 3 traffic passes from one sub network to another. Layer 3 provides for a “Broadcast Domain” to limit broadcasts to a defined domain.
Layer 2 networks do not provide this isolation. Isolation can be critical when malfunctioning systems generate broadcast packets, especially packets that elicit responses that are also broadcast (broadcast storm). A broadcast storm can be devastating to a network. Often, the only way to recover is to completely power-off all devices on the network. Layer 2 architecture provides no protection against broadcast storms, the entire access network serving all subscribers would be affected. Layer 3 switches, on the other hand, provide built-in isolation; and broadcast storms are limited to a single subscriber. The author concedes that there is the opportunity to create a “Broadcast Domain” in Layer 2 Networks using VLAN’s. However, many network operators do not like the complication of VLAN’s or Trunks.
In addition to better manageability, Layer 3 switching also offers greater network performance. Advances in silicon technology have allowed manufacturers to build Layer 3 switches that can forward packets at wire speed, and Layer 3 isolation significantly limits the overhead created by Layer 2 discovery protocols. Flat Layer 2 networks waste significant percentages of bandwidth due to simple broadcasts.
Unlike systems-built decades ago, current Layer 3 routing switches can readily implement forwarding in hardware. Commercially available silicon can forward packets at full wire speeds on Gigabit and 10 Gigabit interfaces.
An area in which Layer 2 and Layer 3 architectures are not equal is in the network overhead they create in a large network. Elements on a network extensively use discovery protocols such as the Address Resolution Protocol (ARP) to learn the identity of their neighbors. In a Layer 2 network, a neighbor can be any other system on the network, while on a Layer 3 network the neighbor is almost always the next hop router. Furthermore, discovery protocols like ARP are broadcast protocols. Layer 2 switches do not isolate broadcasts, instead flooding them throughout the network. In a Layer 2 network, whenever any device tries to find another, the network transmits the discovery packets to every device on the network. Layer 3 switches, on the other hand, confine broadcasts to a single sub network and map their connectivity.
Consider that bandwidth is consumed by ARP broadcasts as the number of network subscribers grows. The percentage of bandwidth consumed depends on traffic patterns in the network. Peer-to-peer traffic is the worst, while pure client/server represents the absolute minimum of overhead. For all types of traffic, overhead on a Layer 3 network is constant at 0.0035 percent of the subscriber’s bandwidth. However, for Layer 2 networks, even in the best case, ARP overhead can reach 35 percent of the subscriber’s bandwidth.
Layer 3 networks simply scale better than Layer 2 ones. The fundamental reason for this advantage is the built-in partitioning that a Layer 3 network creates. Layer 2 networks are flat networks: they have no hierarchy. Layer 3 architectures, on the other hand, create a hierarchy of sub networks.
A Layer 2 network has only one dimension in which it can grow:
Each new device adds to the overall size of the logical Layer 2 network. However, a Layer 3 network can grow in two dimensions. It can expand the number of sub networks or it can increase the number of devices in each sub network. In both cases, the effects of growth are limited. The addition of more sub networks has little impact on the sub networks already present, and growth within a sub network affects only the devices within that sub network.
The other advantages of Layer 3 networks become more and more evident as the network grows and they all contribute to Layer 3’s superior scalability. Consider the effect of each of these qualities as a network increases in size:
- IP addresses and sub networks provisioned automatically by standard, off-the-shelf tools, vs. MAC addresses and VLAN identifiers manually tracked and configured.
- Management and services administered consistently across multiple technologies vs. technology-specific management and services restricted to single networks.
- Troubleshooting and fault isolation using standard, off-the-shelf tools that automatically identify elements with operator-assigned names vs. trouble-shooting limited to cryptic hexadecimal identifiers.
- Inherent and automatic isolation of faulty devices to their own network segment vs. the possibility of a single device disrupting an entire network.
- Network overhead that remains constant as the network grows vs. overhead that increases substantially — and nonlinearly — with network size.
Layer 3 architecture super imposes an automatic network hierarchy. It is this hierarchy that ultimately leads to much better scalability. The strongest evidence for this benefit is the World Wide Web. After all, the Internet is Layer 3 architecture. In conclusion – the design of networks should not assume the denial of peer to peer traffic, nor the transition to IPv6. (also, layer 3).
Our Next Article will delve into the problems using Spanning Tree.
Visit our web site: www.DYMEC.com