IPsec, an extension of the widely adopted internet protocol (IP), encrypts network communications, protecting data against theft and infiltration. Yet, history gives us a better understanding of IPsec.
In the 1970s and 80s, the US Department of Defense (DoD) developed the IP, laying the foundation for ARPANET, a precursor to the internet.
IP enabled data movement in discrete segments called packets from a source computer to a destination computer. Each packet, or datagram, contained control and user information. The former included instructions to deliver the latter, which was known as the payload. Simply put, the IP encapsulates and routes data packets across IP networks.
IP had its limitations, though. The protocol lacked a mechanism to prevent data packets from being snooped on. TV shows in the past have repeatedly alluded to this flaw, indicating it wasn’t hidden for very long.
To end growing security concerns and prevent data sniffing across IP networks, the internet engineering task force (IETF) proposed IP security (IPsec) in 1995. The protocol has remained in use ever since.
The following guide provides insights into the inner workings of IPSec, its features, specific use cases, and more.
How does IPSec work?
IPsec is a layered security protocol that supports IP versions 6 and 4. There are three main protocols within IPsec: authentication header (AH), encapsulating security payloads (ESP), and internet security association and key management protocol (ISAKMP)
Authentication Header
The authentication header (AH) does two things; it validates IP packets to source data origin and provides connectionless data integrity through hash keys. Hash functions allow arbitrary-sized data to be mapped to fixed values, ensuring confidentiality.
Additionally, the AH layer offers protection against replay attacks. In a replay attack, data is retransmitted or delayed by malicious actors. AH eliminates this problem by sequencing IP packets and discarding obsolete ones.
Encapsulating Security Payloads
Like AH, the encapsulating security payloads (ESP) protocol authenticates sender and recipient identities and detects tampered IP packets, ensuring data integrity. But it is encryption that sets it apart. The ESP layer extends IP packets’ security by encrypting the entire payload.
Internet Security Association and Key Management Protocol
ISAKMP adds to AH and ESP by defining security associations between two network entities or hosts exchanging information. Before data transmission, ISAKMP enables the sender and recipient hosts to decide what type of cryptographic algorithm to encrypt the IP packets with, session duration, network parameters, symmetric keys for decryption, and more.
Notably, the IPsec protocol suite encrypts data exchanges between two hosts (host-to-host), networks (network-to-network), or between a security gateway and host (network-to-host).
Methods of operation
IPSec supports two modes of operation: transport and tunnel.
Transport mode
An IP packet consists of two components: the header, which addresses and routes the packet, and the payload, which contains the data.
However, IP packets in transport mode only have their payload encrypted or authenticated. No changes are made to the header, which includes the source and target IP addresses. This is due to the lack of encryption by default in AH.
Transport mode is best suited for host-to-router network connection management.
Tunnel mode
As a stark contrast to transport mode, tunnel mode encrypts the entire IP packet. Essentially, a new IP header is added on top of the original IP header. This is also how private IP addresses or VPNs work. The IP header, which contains the source and the target IP address, is masked to prevent third-party interception.
IPsec’s tunnel mode is best applied to network-to-network communications, host-to-network communications (remote user access), and host-to-host communications (private chat).
Authentication
Depending on the operating system, IPSec may use one of three cryptic algorithms to perform authentication. They are:
- Rivest–Shamir–Adleman (RSA)
- Elliptic curve digital signature algorithm (ECDSA)
- Pre-shared key (PSK)
Let’s go over each in detail.
Rivest–Shamir–Adleman
Rivest–Shamir–Adleman (RSA) is a public-key cryptosystem named after its inventors Ron Rivest, Adi Shamir, and Leonard Adleman.
A public-key cryptosystem has one public encryption key and one private decryption key. Two large prime numbers and an auxiliary value are used to create the RSA public keys.
Let’s say John wants to send a message to Clara. To encrypt the messages with RSA, John must know Clara's public key, and likewise, to decrypt them, Clara must use her private key. Clara sends her public key to John over a secure pathway so he can reliably send his encrypted messages. Remember, Clara's private key is never disclosed.
Elliptic curve digital signature algorithm
The ECDSA algorithm is a variant of the digital signature algorithm (DSA), a federal information processing standard for digital signatures.
Technically, the ECDSA algorithm relies on the algebraic structure of elliptic curves over finite fields. Elliptic curves may be used for key agreement, digital signatures, or pseudo-random generators, among other things.
For elliptic curve cryptography to work, all participating devices must possess a private and public key pair. A message is signed with a private key by its sender, and the recipient uses the sender's public key to verify its authenticity.
Messages that have been altered on route to the recipient will not pass the signature verification test, as the signature only applies to the original message.
Pre-shared key
In cryptography, a pre-shared key (PSK) is a shared secret that two parties have previously shared through a secure channel.
Wi-Fi encryption standards, including wired equivalent privacy (WEP), Wi-Fi protected access (WPA) and the extensible authentication protocol (EAP) use PSK in their encryption methods. The wireless access points (AP) and the clients share the same authentication key.
A PSK may be a password, a passphrase, or a hexadecimal string.
IPsec versus SSL: What’s the difference?
The IPsec protocol suite operates at the network layer in the open systems interconnection
(OSI) model. The secure sockets layer (SSL), on the other hand, operates at the application layer of the OSI model. Each serves a very different purpose: IPsec encrypts IP packers, while SSL encrypts HTTP traffic and web-based transactions.
