How to Implement Zero-Trust SSH Access with Teleport on Linux

One of the most critical aspects of Linux system administration is securing remote access to servers, while SSH has served as the standard protocol for remote management for decades, the traditional approach to SSH access presents several significant security challenges that administrators must address.

The conventional SSH model relies on static keys distributed across multiple systems (creating sprawl that becomes difficult to track), provides limited visibility into access patterns and session activities, and complicates access management as teams grow.

According to zero-trust security principles, the core problem is relying on credentials that are trusted forever.

Teleport is an open-source access platform that implements zero-trust principles specifically for SSH access (and other protocols as well).

Instead of relying on static SSH keys, Teleport issues short-lived certificates that expire automatically, and it maintains comprehensive audit trails of all sessions - allowing administrators to see not only who accessed which systems, but also what commands were executed during each session.

In this guide, we will examine how to configure Teleport to secure SSH access to your Linux servers using a zero-trust approach (which verifies every access request rather than trusting long-lived credentials).

What is Zero-Trust SSH Access?

The traditional SSH security model assumes that trusted network boundaries and long-term SSH keys are enough to keep systems secure.

Once an SSH key is created and shared with a user, it typically stays valid forever unless an administrator explicitly removes it.

In many environments, there is also little to no visibility into what commands users run during SSH sessions or under what circumstances access was granted.

A zero-trust approach to SSH access changes this model by removing built-in trust and enforcing ongoing verification:

• Identity-based access: Users authenticate with their actual identity (typically integrated with existing identity providers) rather than with static key pairs that can be copied or stolen.
• Short-lived certificates: Access credentials automatically expire after a short time (often hours), removing the risk of permanent credentials lingering across systems..
• Complete audit trail: Every session is recorded and logged, capturing not only connection events but also the commands executed and the context of each access request.
• Fine-grained permissions: Role-based access control (RBAC) defines exactly what users can access such as specific servers, commands, and time windows, rather than a simple all-or-nothing access model.
• Just-in-time access: Users request access only when they need it, instead of holding permanent credentials, ensuring permissions match current responsibilities.

Teleport applies these zero-trust principles by acting as both a certificate authority and an access gateway between users and SSH servers.

Instead of connecting directly to servers, users first authenticate with Teleport, which verifies their identity using configured authentication providers.

Teleport then issues short-lived SSH certificates, proxies the connection, and records the entire session for auditing and compliance.

Prerequisites

Before starting, you'll need:

• Two Ubuntu 24.04 LTS servers with at least 2GB RAM each.
• Root or sudo access on both servers.
• Basic understanding of SSH and Linux system administration.
• Servers should be able to communicate with each other over the network.

For this tutorial:

• Server 1 (teleport-cluster): Will run the Teleport cluster (Auth and Proxy services).
• Server 2 (teleport-node): Target SSH server that we'll secure with Teleport.

Installing Teleport on the Cluster Server

First, let's install Teleport on Server 1, which will act as our cluster.

Step 1: Add Teleport Repository