Al Quds University

Subject : Analysis of Hybrid IPSec SSL VPN

Technology

To: Dr

From: Eng. Rafi Droubi

2008

Abstract

1
 Securing data over the network is hard and complicated issue while
the threat of data modification and data interruption is rising. The goal of
network security is to provide confidentiality, integrity and authenticity. The
combination of these properties is the pillar of the security protocols.
Virtual Private Network VPN technology provides a way of protecting
information being transmitted over a public network like the Internet, by
allowing remote sites or users to establish virtually a "private Tunnel” to
securely reach an internal network, accessing resources, data and
communications. The most commonly used tunneling protocols are IPSec
(IP Security), L2TP, PPTP and SSL Secure Socket Layer). Before a packet is
transmitted, it is encapsulated (wrapped) in a new packet with a new header
to provide routing information, so it can traverse a shared or public network
before it reaches its destination. When the packet reaches the tunnel
endpoint, it is “decapsulated” and forwarded to its final destination. A packet
with a private non-routable IP address can be sent inside a packet with
globally unique IP address, thereby extending a private network over the
Internet.
This paper provides a general overview of VPN and core VPN
technologies and the security considerations that need to be taken into
account when implementing a virtual private network. It also compares the
two tunneling protocols (IPSec and SSL) in terms of characteristic and
functionality. In this paper we also present an analysis of a hybrid IPSec and
SSL methodology as a Hybrid IPSec SSL VPNs Technology system
combines advantages of both IPSec VPN and SSL VPN technologies, but
eliminates the shortcomings.

Introduction

2
 In today’s society organizations and companies are expanding
globally from region to region. Employees working from home offices are
also becoming very popular. With this expansion, organizations develop a
need to communicate with these offices over highly secure, confidential,
reliable connections regardless of the location of the office.
VPNs or Virtual Private Networks are used by many organizations
and companies to fulfill the need to communicate securely and
confidentially over the internet with its employees and offices outside the
corporate network. A VPN creates a private and secure connection, known as
tunnels, throughout systems that use the data communication capability of

an unsecured and public network like Internet. Organizations which decide

to use VPNs as their means of secure communication choose between the
more commonly used IPSec and SSL secure protocols. Both protocols have
their advantages and disadvantages; the deciding factors between the two
depend on the infrastructure of the corporate network, its specific security
requirements, costs, and reliability.

What is VPN?
VPN stands for Virtual Private Network, and relates to a set of
software and hardware that enable your computer to transparently connect to
a remote network as if you were physically attached to that network. In
addition, the communication between your computer and the remote VPN
hardware is encrypted. A common example would be an employee accessing
information at the office from home using his or her Internet connection.

3
 VPN transmits data by means of tunnelling. Before a packet is
transmitted, it is encapsulated (wrapped) in a new packet, with a new header.
This header provides routing information so that it can traverse a shared or
public network, before it reaches its tunnel endpoint. This logical path that
the encapsulated packets travel through is called a tunnel. When each packet
reaches the tunnel endpoint, it is “decapsulated” and forwarded to its final
destination. Both tunnel endpoints need to support the same tunnelling
protocol. Tunnelling protocols are operated at either the OSI (Open System
Interconnection) layer two (data-link layer), or layer three (network layer).

Earlier generations of VPN technology were “narrow” solutions that

supported limited networking protocols, utilizing transport mechanisms
including PPTP (Point to Point Tunneling Protocol), L2TP (Layer 2
Tunneling Protocol), and today’s pre-dominant IPSec (Internet Protocol
Security).
Benefits of using VPN technologies
Using of virtual private networks achieves many benefits summarized
as follows:
1) Lower Costs: combining Internet, Intranet, and Extranet connectivity
through the same VPN solution reduces the cost and complexity of
managing multiple networks.
Combining multiple devices onto a single platform also
reduces the total hardware and software cost. Integration
of packaging and power supplies reduces duplication and
allows common tasks and data to be shared. Inter-
component interfaces can be eliminated (or at least can
be built directly into the software). An integrated VPN
platform also requires less physical space, less power and
less cabling, all of which result in lower total cost of

4
 operation. An integrated platform will also be covered by
a single vendor’s support and maintenance contract,
providing the savings from “one-stop shopping.

2) Extended Geographic Connectivity: a VPN connects remote workers

to central resources, making it easier than ever to set up widely
distributed global operations.
3) Increased Return on Investment: an effective security solution
significantly reduces threats and consequently decreases downtime
and lost business.
4) Easily scalable: a VPN allows customers to utilize the remote access
infrastructure within internet service providers (ISPs) and companies
can add a virtually unlimited amount of capacity without adding
significant infrastructure .

Overview of IPSec
IPsec was developed by IETF (the Internet Engineering Task Force) for
secure transfer of information at the OSI layer three across a public
unprotected IP network, such as the Internet. IPsec enables system to select
and negotiate the required security protocols, algorithm(s) and secret keys to
be used for the services requested. IPsec provides basic authentication, data
integrity and encryption services to protect unauthorized viewing and
modification of data. It makes use of two security protocols, AH
(Authentication header) and ESP (Encapsulated Security Payload), for
required services. However, IPsec is limited to only sending IP packets.

5
1. ESP (Encapsulated Security Payload) protocol provides data

confidentiality, in addition to source authentication and integrity. ESP

uses symmetric encryption algorithms, such as 3DES, to provide data
privacy. The algorithm needs to be the same on both communicating
peers. ESP can also support encryption-only or authentication-only
configurations.

2. AH(Authentication Header) protocol provides source authentication, and

integrity of IP packets, but it does not have encryption. An AH header

added to the IP packet contains a hash of the data, a sequence number
etc., and information that can be used to verify the sender, ensure data
integrity and prevent replay attacks.

Modes of Operation
Each security protocol supports two modes of operation: a tunnel
mode and a transport mode.
• Tunnel mode: the original IP packet, with original destination address,
is inserted into a new IP packet and the AH and ESP are applied to the
new packet. The new IP header points to the end point of the tunnel.
Upon receipt of the packet, the tunnel end point will decrypt the
content and the original packet is further routed to its final destination
in the target network.

6
 • Transport mode: the AH and ESP headers are applied to the data of
the original IP packet. The mode encrypts and / or authenticates the
data but not the IP header. The overhead added is less than that
required in tunnel mode. However, the final destination and source
addresses could be sniffed. Attackers can perform traffic analysis
based on header information in this type of header. It is generally only
used for host-to-host connections.

Overview of SSL
SSL is a transport-layer protocol that use TCP port 443.There are a
number of cryptographic features provided by SSL and these include
confidentiality, integrity, and digital signatures. Unlike IPSec, in which the
two communicating parties agree to cryptographic functions, SSL uses
cipher suites to define the set of cryptographic functions for a client and
server to use when communicating.
The uses for SSL are endless. Any application that needs to transmit data
over an unsecured network such as the Internet or a company intranet is a
potential candidate for SSL. SSL provides security, and more importantly,
peace of mind. When using SSL, you can be fairly sure that your data are
safe from eavesdroppers and tampering.
SSL protects the communications channel. It also provides
authentication (on the client side, optionally on the server side) of
communicating parties. SSL can secure any connection between two points,

7
and no one monitoring the connection can do anything destructive or gain
unauthorized access to any sensitive information.
SSL provides a secure channel without the need for either end to meet to
exchange keys. SSL is to secure communications as TCP is to normal
communications—it provides a standard communications infrastructure that
compliant applications can use easily and nearly invisibly.
SSL provides a vitally important component of any secure system. Basic
authentication mechanisms such as the Telnet password and basic HTTP
authentication become very powerful security options when executed using
SSL instead of plain TCP—passwords are no longer sent plain-text, making
these methods much more useful. SSL encrypts the connection, not the data
at either end, and does not contain any mechanism for user authentication or
password protection (only the connection is authenticated—the security fails
if the machine at either end is compromised).

An SSL VPN gateway can authenticate itself to the Web user using a SSL
server certificate signed by a trusted CA (Certification Authority), in order
that the user can verify that he / she is talking to a trusted server via their
browser. In practice, some SSL VPNs may use a self-signed digital
certificate that is not normally trusted in most web browsers. In this case,
the user might need to add the SSL VPN's server certificate to the user's own
list of trusted certificates, or accept „'Yes' to trust the certificate.

Differences between IPSec and SSL

Each of the protocols has unique properties. Choosing IPSec or SSL

depends on the security needs. If a specific service is required and is

8
supported by SSL, it is better to select SSL. If over all services or Gateway-
to-Gateway communications are needed then IPSec is a good choice
considering the following:
IPSec uses a shorter form of HMAC than SSL, thus SSL data integrity is
more secured. SSL is more compatible with firewall than IPSec, unless
IPSec and Firewall are integrated in the same device. Unlike SSL, IPSec
clients need special IPSec software for remote access. In low bandwidth
networks or dialup networks using compression is beneficial, SSL doesn’t
support that. Pre-Shared scheme is easier to configure and doesn't require
any PKI infrastructure, IPSec supports compression but unfortunately SSL
doesn’t support it. IPSec is capable of protecting wireless networks. In most
cases IPSec doesn't interoperate well so both sides of the connection are
required to have the same vendors devices.

VPN technologies
Several different technologies are used to create security facility to
VPNs. Figure (1) illustrate the applications of two technologies used to
provide remote VPN access, Internet Protocol Security (IPSec) VPN and
Secure Sockets Layer (SSL) VPN.

9
 Figure (1)
There are many advantages and disadvantages for the IPSec VPN
among them are:

1) Advantages: network to network communication, desk-like network

access experience and protocol independent.
2) Disadvantages: it does not work through firewalls, difficult to
deploy, maintain, and manage, high cost of support and troubleshooting,
client IP addresses are visible from the accessed network and network
bridging allows network viruses and worms traversal .

On the other hand, SSL is a protocol used to secure web based

communications over the internet at the application layer. It uses encryption
and authentication to keep communications private between two devices,
which are typically a web server and a user machine. Like IPSec, SSL also
provides flexibility in allowing enterprises to define the level of security that
best meets their needs.
SSL VPN allows users to remotely access important enterprise applications,
systems, and files from any Internet enabled computer. It enhanced to

10
provide users with secure remote access to internal resources, and promises
to improve both mobile user's productivity and convenience.
There are many advantages and disadvantages for the SSL VPN among them
are:
1) Advantages: easy to deploy, clientless and client IP addresses are
not visible from the accessed network.
2) Disadvantages: works only for web applications that do not use
many of the more popular features like ActiveX controls and Java applets. It
supports the native corporate applications require creating custom
connection and degraded performance of client-server and web applications.

IPSec VPNs
IPSec (a network-layer VPN protocol) can offer organizations an easy,
cost-effective way to route communications between fixed sites, delivering
high performance connectivity and resiliency to match the needs of the most
demanding network environments. IPSec VPNs were created as a cost-
effective encrypted transport alternative to private or leased lines enabling
organizations to use the Internet infrastructure to extend the private network
across geographically distributed locations.

Technically, network-layer VPNs address the challenge of how to use the

Internet (which uses the IP protocol, and usually transmits text in the clear)
as a transport for sensitive, Multiprotocol traffic. Network-layer VPNs
provide a combination of encryption and tunneling functions to meet these
challenges. They use negotiation protocols, like ISAKMP to authorize
remote peers and tunneling protocols like ESP and AH to encapsulate user
data within an IP “wrapper” that will traverse the Internet. This encapsulated

11
data is received by the network layer VPN gateway, “unwrapped,”
decrypted, and forwarded to the recipient. Traffic arriving from the VPN
gateway is handled as if it originated from any user within the LAN itself.
As a result, network-layer VPNs provide users the same, continuous access
to the network that they would have if they were physically located at the
same facility. This is ideal for facilitating regular communications and
resource sharing among users at geographically separate offices to improve
productivity agency-wide.

SSL VPN
The term SSL VPN is used to refer to a new and fast-growing product
category comprised of a variety of technologies, based on the Secure
Sockets Layer (SSL) protocol. To broadly define what products and
technologies are within this category, one can begin with the term “VPN”
itself. VPN, or Virtual Private Network, refers to the practice of using a
public network like the Internet to transmit private data. Until 2001, most in
IT did not add a descriptor to VPN because almost all VPNs available at that
time used some type of network-layer transport. The early standard in the
VPN space was the IP Security Protocol (IPSec), although some vendors use
other methods, including Layer 2 Tunneling Protocol (L2TP), and Point-to-
Point Tunneling Protocol (PPTP). SSL VPNs use a different methodology to
transport private data across the public Internet. Instead of relying upon the
end user to have a configured client on an agency-managed computer, SSL
VPNs use SSL /HTTPS which is the secure transport mechanism built-in to
all standard Web browsers. Using an SSL VPN, the connection between the
user and the internal resource occurs via an HTTPS connection at the

12
 application-layer, as opposed to IPSec VPN’s “tunnel” at the network-layer.
SSL VPNs still utilize the robust security of 3DES encryption, but they don’t
require an “open pipe” to be established back to the internal resources.
Instead, SSL VPNs are application and user aware, so access is granted only
to the resources designated by the administrator on a per user basis. To ease
the configuration of policy, the administrator can define roles which
establish access privileges used by numerous users. Then each user is
assigned to a role.

IPSec VPN versus SSL VPN

Table (1) shows the technical comparison between IPSec VPN and
SSL VPN technologies
Table (2.2) shows the functional comparison between IPSec VPN
and SSL VPN technologies .

Table 1- The technical comparison between IPSec and SSL VPNs

NO. Optional Clause IPsec SSL

1. Authentication Digital Signature and Secret Key Digital Signature

Algorithm Algorithm
2. Authentication Method One type of Authentication Many types of
Authentication
3. MAC( Message
Authentication Code) HMAC-SHA-1-96 HMAC-SHA-1
HMAC-MD5
HMAC-MD5-96
Hash Length 12 Byte , 12 Byte 20 Byte , 16 Byte

4. Around Transport • Over UDP • Over TCP

Layer • Fixed port • Port can be
• allow UDP or TCP based changed
applications work with(by • Supporting only

13
 adding a new TCP header TCP
to the original packet) application is a
shortcoming of SSL

5. Connection Mode Tunnel Mode One connection per one

Transport Mode session type

6. Order of Cryptographic Encrypt data then create MAC Create MAC then
Operations encrypt data

7. Cipher List Proposal Bi Direction One Direction

7. Interoperability IPSec doesn't integrate well trouble free and well

with other IPSec of other integrated
Vendor
8. Overhead Size Add extra size to the original SSL needs less overhead
packet
Tunnel ESP : 32 Byte HMAC-MD5 : 21 Byte
Mode ESP & AH : 44 Byte HMAC-SHA-1 : 25 Byte

Transport ESP : 36 Byte

Mode ESP & AH : 48 Byte
9. Residing Layer Network layer Application layer

10. Time of Handshake Main Mode (PSK) 97 msec Server Auth. 41.7 Msec
Process
Main Mode (RSA) 170 msec Client Auth. 74.8 Msec

Aggressive Mode 56 msec Server (DH) 66.1 msec

(PSK)
Client (DH) 118.6
msec

11. Compression Algorithm Compression is utilized by IPSec compression is used in a

through a compression protocol small range with SSL,
called IPComp only OpenSSL supports
compression.
12. Performance

Table 2 - The functional comparison between IPSec and SSL VPNs

14
NO. Function IPSec SSL
1. Configuration Hard Easy
2. Client Authentication Must Option
3. Pre-Shared Key Yes No
4. Interoperability Problem Yes No
5. TCP Application Support All Some
5. UDP support Yes No
7. Throughput Rate High High
8. Compression Support Yes Open SSL
9. Handshake Time Slow Fast

1IPSec or SSL VPN

Many users are struggling to decide which technology should be
deployed where. Where do IPSec and SSL VPNs fit into your network
security posture, and which problems can each technology best address?
What is required to deploy and administer an IPSec or SSL VPN?

This confusion is not mitigated by the fact that most debates over IPSec and
SSL have largely focused on the technical details of the protocols rather than
the usage scenarios. The fact is that IPSec and SSL are not mutually
exclusive technologies, and both technologies are equally secure in terms of
bit transport over an insecure network. They can – and in fact, often are –
deployed simultaneously in the same organization. The deciding factor
between them lies not in what each protocol provides, but in what each
deployment is designed to accomplish. When one considers the cost/benefit
of each type of deployment, as well as what problems each technology was
designed to address, the deployment choices become clearer.

15
3Hybrid IPSec SSL VPNs Technology
0Given the limitations of IPSec and SSL VPN technologies, vendors have
naturally endeavored to develop technologies that mix the benefits of both
protocols, while eliminating the negatives. Rather than melding the SSL and
IPSec protocols, some manufacturers have attempted to provide the benefits
of both IPSec and SSL via their own proprietary protocol. Others have
supplemented and enhanced their IPSec or SSL products.
4As might be expected, today’s hybrid VPNs attempt to provide all of the
benefits of IPSec and SSL VPNs, while removing all of the burdens.
5

16
What level of access control is required?
If you want to grant access to all applications and resources
within a network segment to all users in the IP-VPN, an IPsec VPN will
serve well.

If you want to control access to specific applications—

particularly for Web-accessible applications only, SSL might be the
better choice, because it secures communications from user to
application server, rather than from user to site gateway.

If you want variable levels of access control—( HYBRID )say, one

level of access privileges when the user is in the branch office and
another when the user is accessing the network from trade shows,
consider using IPsec for the in-office access and SSL for on-the-road
access—or run SSL over IPsec for controlled and end-to-end security.

What roles will your users have?

If users simultaneously have confidential and non-confidential roles such as

executives who need
to download customer presentations from their hotel rooms and confidential
financial reports from
their offices, a combination of SSL and IPsec access can provide the premium
flexibility needed in
one role and the premium security needed in the other.

Which is more important: fast deployment or future scalability?

If it is critical to get up and running quickly
If it’s more critical to easily add and change applications later,

If both considerations are important, consider deploying SSL for basic e-mail,
file-sharing, and
intranet access to quickly meet immediate access needs while rolling out IPsec
VPN capabilities.

What’s the bottom line?

Enterprise networks are by nature heterogeneous. VPN access to corporate
resources must serve a
broad range of user requirements that change from moment to moment, or day
to day. Both IPsec and
SSL protocols have their merits. Both are effective, standards-based choices for
deploying secure
remote access. Both have advantages and limitations, depending on the
circumstances—the
applications, users, security, confidentiality, and deployment considerations in
question. There is no

17
‘either-or’ path. For these reasons, most enterprises will benefit from the
deployment of both SSL and
IPsec-based VPNs.

18
Reference:
1.A Technical Comparison of IPSec and SSL, AbdelNasir .
Alshamsi,Takamichi Saito,Tokyo University of Technology>
2. www_stunnel_org
3. IPSec and SSL VPN Decision Criteria
4. VPN SECURITY February 2008

19
20