├── .gitignore ├── README.md ├── access-control-and-rootly-powers ├── data │ └── sudo-sesame.png └── readme.md ├── booting-and-system-management-daemons ├── data │ ├── boot-process.png │ └── subcommand-systemd.png └── readme.md ├── cloud-computing ├── data │ └── cloud-providers.png └── readme.md ├── config-management ├── data │ ├── a-vs-s.png │ ├── ansible-play.png │ ├── ansible_diagram.png │ ├── cm-tools.png │ ├── new-client.png │ └── terminology.png ├── readme.md └── training │ └── bindings-examples.md ├── containers ├── data │ ├── bridge-net-docker.png │ ├── docker-archi.png │ ├── docker-group-issue.png │ ├── sched-arch.png │ ├── storage-drivers.png │ └── ufs-docker.png ├── readme.md └── training │ ├── mesos.pdf │ └── modern-guide-to-container-monitoring-and-orchestration.pdf ├── continuous-integration-and-delivery ├── data │ ├── ci-cd-pipeline.png │ ├── cont-based-deploy.png │ ├── example-app.png │ └── rc-release.png ├── readme.md └── training │ └── CI-CD-Pipelines-Guide.pdf ├── dns ├── data │ ├── bind-statements.png │ ├── dns-delegation.png │ ├── dns-example.png │ ├── dns-record-types.png │ ├── dns-zone-tree.png │ ├── ns-taxonomy.png │ ├── sec-feat-bind.png │ ├── signed-zone.png │ └── srv-atrust.png ├── readme.md └── training │ ├── dns-e2e-transfer.md │ ├── dns-trace.py │ ├── rec-vs-norec.md │ ├── registrar-regsitry.md │ ├── reverse-dns.md │ └── rfc-1033.md ├── docker-compose.yml ├── drivers-and-the-kernel ├── data │ ├── components-device-file.png │ ├── kernel-dev-drivers.png │ └── udevd-match-keys.png └── readme.md ├── electronic-mail ├── data │ ├── dsn-error-codes.png │ ├── evi-and-david.png │ ├── mail-sys-comp.png │ ├── mta-market.png │ ├── postfix-arch.png │ └── smtp-cmd.png └── readme.md ├── ip-routing ├── data │ ├── B-rtb.png │ ├── R1-rtb.png │ ├── host-rtb.png │ └── packet-forwarding.png └── readme.md ├── lab-volumes ├── Dockerfile.centos ├── Dockerfile.debian ├── centos │ └── test └── debian │ └── test ├── logging ├── data │ ├── case-study-logging.png │ ├── common-action.png │ ├── facility.png │ ├── level-qualif.png │ ├── log-files.png │ ├── logrotate-options.png │ ├── rsyslog-conf.png │ ├── rsyslog-prop.png │ └── severity.png └── readme.md ├── monitoring ├── data │ ├── carbon-summarization.png │ ├── commercial-platforms.png │ ├── graphite.png │ └── snmpwalk.png ├── readme.md └── training │ ├── cpt93_configuration_chapter_010011.pdf │ └── net-snmp.pdf ├── network-file-system ├── data │ ├── client-specs.png │ ├── nfs-joke.jpg │ └── nfs-versions.png ├── readme.md └── training │ ├── aws-efs.pdf │ └── root-squash.md ├── performance-analysis ├── data │ ├── Virtual-vs-Physical-adresses.png │ ├── lru-workflow.png │ ├── my-linux-topo.png │ └── performance-analysis-flow.png ├── readme.md └── training │ └── linux_utemezes.pdf ├── physical-networking ├── data │ └── evolution-of-eth.png └── readme.md ├── printing ├── data │ └── gutenberg printing press.png └── readme.md ├── process-control ├── data │ ├── process-explanation.png │ └── process-information.png ├── readme.md └── training │ ├── cron.md │ └── current-date.sh ├── scripts-and-shell ├── readme.md └── training │ ├── awk.md │ ├── backup_fn.sh │ ├── omit.awk │ └── showusage.sh ├── security ├── data │ ├── client-config.png │ ├── config-files.png │ ├── hash-confirm.png │ ├── port-forwarding.png │ └── ssh-agent-forwarding.png ├── readme.md └── training │ ├── OPENVPN in details.pdf │ ├── details-view.png │ ├── list-view.png │ ├── params-free.png │ ├── setting-openvpn.md │ └── vpn-test │ ├── docker-compose.yml │ └── webserver │ └── html │ └── index.html ├── single-sign-on ├── data │ ├── SSO-components.png │ ├── common-attrs.png │ ├── pam-example.png │ └── sssd-conf-file.png ├── readme.md └── training │ └── genuine-dialog.md ├── smb ├── data │ ├── carnaval-mindelo-defile-ecole-samba-tropicale-1620x600.jpg │ ├── danse-samba.jpg │ ├── smb-history.png │ └── smb-vs-nfs.png ├── readme.md └── training │ ├── sambaxp-2015-cloudy-future.pdf │ └── smb-hands-on │ ├── Dockerfile │ ├── Dockerfile.clients │ ├── docker-compose.yml │ ├── eng.conf │ ├── homes.conf │ └── readme.md ├── software-installation ├── data │ ├── PXE_Boot.png │ └── intel-uefi-pxe-boot-performance-analysis.pdf ├── readme.md └── training │ └── pxe.md ├── storage ├── data │ ├── cow.png │ ├── fdisk-recipe.png │ ├── fs-space-management.png │ ├── hdd-vs-ssd.png │ ├── logical-vol-capacities.png │ ├── lvm-commands-linux.png │ ├── raid-0.png │ ├── raid-1.png │ ├── raid-10-01.png │ ├── raid-5.png │ ├── raid-6.png │ ├── storage-mgmt-layer.png │ ├── trad-part-scheme.png │ ├── vgdisplay-out.png │ ├── vgdisplay-out2.png │ └── zfs-archi.png ├── readme.md └── training │ └── YM_RAID_Primer.pdf ├── tcp-ip-networking ├── data │ ├── etc-hosts.png │ ├── iptables-flags.png │ ├── netmask.png │ ├── syn-ack.png │ └── tcp-ip_layer_model.png ├── readme.md └── training │ └── iptables.md ├── the-filesystem ├── data │ ├── file-type-encoding.png │ ├── nsfv4.png │ ├── pathnames.png │ └── permissions-encoding.png ├── readme.md └── training │ └── x-windows.md ├── update_read_status.sh ├── user-management ├── data │ ├── command-and-config.png │ └── common-scripts.png ├── readme.md └── test ├── virtualization ├── data │ ├── containerization.png │ ├── type1-vs-type2.png │ ├── vm-vs-contd.png │ └── xen-comp-dom0.png ├── readme.md └── training │ ├── Namespaces_Cgroups_Containers.pdf │ └── TorreyGuestLecture-Hypervors.pdf ├── web-hosting ├── data │ ├── apache-conf.png │ ├── cache-layers.png │ ├── cdn-work.png │ ├── common-headers.png │ ├── components-web-stack.png │ ├── ha-proxy.png │ ├── http-response.png │ ├── http-server-types.png │ ├── load-balancer.png │ ├── nginx-conf-details-platform.png │ ├── open-source-caching.png │ └── subdirs-debian-apach.png ├── readme.md └── training │ ├── all_about_load_balancing-wp-en.pdf │ └── reverse-proxy-cache.pdf └── where-to-start ├── data ├── other-sources.png └── table-of-linux-distros.png └── readme.md /.gitignore: -------------------------------------------------------------------------------- 1 | .obsidian/ 2 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Unix-and-Linux-sysadmin-notes 2 | 3 | Unix and Linux system administration handbook by Evi Nemeth Garth Snyder Trent R. Hein Ben Whaley Dan Mackin 4 | 5 | ![book cover](https://crescentvale.com/wp-content/uploads/2017/12/unix-and-linux-system-administration-handbook-3.jpg) 6 | 7 | You can buy the book [here](https://www.amazon.com/UNIX-Linux-System-Administration-Handbook/dp/0134277554) 8 | 9 | ## POV 10 | 11 | This repository is a collection of notes from the book *Unix and Linux system administration handbook by Evi Nemeth Garth Snyder Trent R. Hein Ben Whaley Dan Mackin*. I am reading the 5th edition of the book. This shit is a comprehensive guide to Unix and Linux system administration. 12 | 13 | ## Path to the notes 14 | 15 | The notes are organized in chapters. Each chapter has a `readme.md` file that contains the notes for that chapter. The notes are organized in a way that makes it easy to follow the book. I am not following the chapters in order. I am reading the book in a way that makes sense to me. 16 | 17 | ### List of chapters: 18 | 19 | - [Chapter1: Where to start](./where-to-start/readme.md) 20 | - [Chapter2: Booting and System Management Daemons](./booting-and-system-management-daemons/readme.md) 21 | - [Chapter3: Access Control and Rootly Powers](./access-control-and-rootly-powers/readme.md) 22 | - [Chapter4: Process Control](./process-control/readme.md) 23 | - [Chapter5: The Filesystem](./the-filesystem/readme.md) 24 | - [Chapter6: Software Installation and Management](./software-installation/readme.md) 25 | - [Chapter7: Scripting and Shell](./scripts-and-shell/readme.md) 26 | - [Chapter8: User Management](./user-management/readme.md) 27 | - [Chapter9: Cloud Computing](./cloud-computing/readme.md) 28 | - [Chapter10: Logging](./logging/readme.md) 29 | - [Chapter11: Drivers and the Kernel](./drivers-and-the-kernel/readme.md) 30 | - [Chapter12: Printing](./printing/readme.md) 31 | - [Chapter13: TCP/IP Networking](./tcp-ip-networking/readme.md) 32 | - [Chapter14: Physical Networking](./physical-networking/readme.md) 33 | - [Chapter15: IP Routing](./ip-routing/readme.md) 34 | - [Chapter16: DNS - The Domain Name System](./dns/readme.md) 35 | - [Chapter17: Single Sign-On](./single-sign-on/readme.md) 36 | - [Chapter18: Electronic Mail](./electronic-mail/readme.md) 37 | - [Chapter19: Web Hosting](./web-hosting/readme.md) 38 | - [Chapter20: Storage](./storage/readme.md) 39 | - [Chapter21: The Network File System](./network-file-system/readme.md) 40 | - [Chapter22: SMB - Server Message Block](./smb/readme.md) 41 | - [Chapter23: Configuration Management](./config-management/readme.md) 42 | - [Chapter24: Virtualization](./virtualization/readme.md) 43 | - [Chapter25: Containers](./containers/readme.md) 44 | - [Chapter26: Continuous Integration and Delivery](./continuous-integration-and-delivery/readme.md) 45 | - [Chapter27: Security](./security/readme.md) 46 | - [Chapter28: Monitoring](./monitoring/readme.md) 47 | - [Chapter29: Performance Analysis](./performance-analysis/readme.md) 48 | - [Chapter30: Data Center Basics](./data-center-basics/readme.md) 49 | - [Chapter31: Methodology, Policy, and Politics](./methodology-policy-and-politics/readme.md) 50 | 51 | ## Testing lab 52 | 53 | I've set up a testing lab to test the concepts discussed in the book. You will need to install docker and docker-compose to run the lab. 54 | 55 | To install docker and docker-compose, follow the instructions [here](https://docs.docker.com/get-docker/) 56 | 57 | To run the lab, clone the repository and run the following command: 58 | 59 | ```bash 60 | docker compose up -d --build 61 | ``` 62 | 63 | This will create two containers, one for the Debian server and the other for the CentOS server. You can access the containers using the following command: 64 | 65 | ```bash 66 | docker exec -it lab-[debian|centos] bash 67 | ``` 68 | 69 | You will then be logged into the container. You can then run the commands discussed in the book to test the various concepts. 70 | 71 | A volume is mounted to the `lab-[debian|centos]` container. You can use this volume to share files between your host machine and the container. 72 | 73 | ## Why I am reading this book 74 | 75 | I am reading this book to learn more about Unix and Linux system administration. As a DevOps engineer, it's a must to know how to manage Unix and Linux systems. 76 | -------------------------------------------------------------------------------- /access-control-and-rootly-powers/data/sudo-sesame.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/abdoufermat5/unix-and-linux-sysadmin-notes/10b44b43dff4235fd3e4e9755d6f60361093b1e9/access-control-and-rootly-powers/data/sudo-sesame.png -------------------------------------------------------------------------------- /access-control-and-rootly-powers/readme.md: -------------------------------------------------------------------------------- 1 | # Chapter 3: Access Control and Rootly Powers 2 | 3 | ![sudo-sesame](./data/sudo-sesame.png) 4 | 5 | ## Standard UNIX Access Control 6 | 7 | The standard UNIX access control model has remained largely unchanged for decades. With a few enhancements, it continues to be the default for general-purpose OS distributions. The scheme follows a few basic rules: 8 | 9 | - Access control decisions depends on which user is attempting to perform an operation, or in some cases, on that user's membership in a UNIX group. 10 | - Objects (e.g files and processes) have owners. Owners have broad (but not necessarily unrestricted) control over their objects. 11 | - You own the objects you create. 12 | - The special user account called "root" can act as the owner of any object. 13 | - Only root can perform certain sensitive administrative operations. 14 | 15 | ## Filesystem access control 16 | 17 | Every file has both an owner and a group, sometimes referred to as the "group owner." 18 | 19 | Both the kernel and filesystem track owners and groups as numbers rather than text names. In the most basic case, user identification number (UIDs) are mapped to usernames in the /etc/passwd file, and GIDs are mapped to group names in the /etc/group file. 20 | 21 | 22 | ## The root account 23 | 24 | The root account is the most powerful account on a UNIX system. It has the ability to bypass all access control checks and can perform any operation on the system. The root account is also known as the superuser account. 25 | 26 | The UID of the root account is always 0. This is hard-coded into the kernel and is not configurable. The root account is the only account with a UID of 0. 27 | 28 | Traditional UNIX allows the superuser (that is, any process for which the effective UID is 0) to perform any valid operation on any file or process. “Valid” is the operative word here; certain operations (such as executing a file on which the execute permission bit is not set) are forbidden even to the superuser. 29 | 30 | Some examples of restricted operations include: 31 | 32 | - Creating device files in /dev 33 | - Setting the system clock 34 | - Raising resource usage limits and process priorities 35 | - Setting the system's hostname 36 | - Configuring network interfaces 37 | - Shutting down the system 38 | 39 | Again superuser is not the root account, but any process with an effective UID of 0. This is an important distinction because it means that the superuser is not necessarily the root account. For example, a setuid root program is a program that runs with the effective UID of root, but it is not the root account. This is a subtle but important distinction. 40 | 41 | An example of superuser powers is the ability of a process owned by root to change its UID and GID. The login program and its GUI equivalents are a case in point; the process that prompts you for your password when you log in to the system initially runs as root. If the password and username that you enter are legitimate, the login program changes its UID and GID to your UID and GID and starts up your shell or GUI environment. Once a root process has changed its ownerships to become a normal user process, it can’t recover its former privileged state. 42 | 43 | ## The setuid and setgid execution 44 | 45 | The setuid and setgid bits are a special type of permission that can be set on executable files. When an executable file has the setuid bit set, the file will always be executed with the effective UID of the file's owner. When an executable file has the setgid bit set, the file will always be executed with the effective GID of the file's group owner. 46 | 47 | Programs that run setuid, especially those that run setuid root, are a potential security risk. If a setuid root program has a security hole, an attacker can exploit the hole to gain root access to the system. For this reason, many UNIX systems are configured to ignore the setuid bit on programs that are run from file systems that are mounted with the noexec option. 48 | 49 | ## Management of the root account 50 | 51 | ### su: substitute user identity 52 | 53 | A better way to become root is to use the su command. The su command allows you to become another user, including root. When you run su, you are prompted for the password of the user you want to become. If you enter the correct password, you become that user. If you run su without specifying a username, you become root. 54 | 55 | **su** doesn't record the commands executed as root in the system logs. This makes it difficult to track who did what as root, but it does create a log netry that states who became root and when. 56 | 57 | It's a good habits to the full pathname to su (e.g. **/bin/su** or **/usr/bin/su**) to avoid running a trojan horse version of su that an attacker might have installed in your PATH. 58 | 59 | On most systems, you must be a member of the group *wheel* to use **su**. 60 | 61 | We use **sudo** as a more secure alternative to **su**. **su** is best reserved for emergency use when **sudo** is not available. 62 | 63 | ### sudo: limited su 64 | 65 | If the root account is used by several administrators, it can be difficult to track who did what as root. 66 | 67 | The most widely used solution to this problem is the sudo program. sudo allows you to run a command as another user, including root. The sudo program logs all commands that are run as root, so it is easy to track who did what as root. 68 | 69 | **sudo** takes as its argument a command line to be executed as root (or as another restricted user). **sudo** consults the file **/ets/sudoers/**, which lists people who are allowed to use **sudo** and what commands they are allowed to run. If the proposed command is allowed, **sudo** asks for the user's password and then runs the command. 70 | 71 | The **sudoers** file is edited with the **visudo** command, which checks the syntax of the file before saving it. The **visudo** command uses the **EDITOR** environment variable to determine which editor to use. If **EDITOR** is not set, **visudo** uses **vi**. 72 | 73 | **sudo without password** 74 | 75 | If you want to allow a user to run **sudo** without entering a password, you can use the **NOPASSWD** tag. For example, to allow the user **bob** to run **sudo** without entering a password, you can use the following line in the **sudoers** file: 76 | 77 | ``` 78 | bob ALL = NOPASSWD: ALL # Don't try this at home!! 79 | ``` 80 | 81 | The most common cases are when performing remote configuration management with tools like **ansible** or **puppet**. 82 | 83 | A better alternative to **NOPASSWD** in the context of remote configuration management is to use **ssh** keys to authenticate to the remote system. This way, the user doesn't need to enter a password to run **sudo**. 84 | 85 | SSH key forwading can be used to authenticate to the remote system using **ssh** keys and then run **sudo** without entering a password. 86 | 87 | The sudo program is maintained by Todd C. Miller and is available from [www.sudo.ws](https://www.sudo.ws/). 88 | 89 | ### Disabling the root account 90 | 91 | Some administrators disable the root account entirely. This is done by setting the root account's password to an impossible value, such as a long string of random characters. This makes it impossible to log in as root, even if you know the root password. 92 | 93 | ## PAM: Pluggable Authentication Modules 94 | 95 | PAM is a system that allows you to configure the authentication process on a UNIX system. PAM is a flexible system that allows you to configure the authentication process in a variety of ways. PAM is used by many UNIX programs, including the login program, the su program, and the sudo program. 96 | 97 | As an example, you can use PAM to configure the login program to require two-factor authentication for certain users, or to require a password and a fingerprint scan for other users. 98 | 99 | ## Kerberos: network cryptographic authentication 100 | 101 | Kerberos is a network authentication system that allows you to authenticate to a network service without sending your password over the network. Kerberos is widely used in large organizations, especially in academic and research institutions. 102 | 103 | Whereas PAM is an authentication framework, Kerberos is a specific authentification method. At sites that use Kerberos, PAM is often configured to use Kerberos for authentication. 104 | 105 | Kerberos use a trusted third party (a server) to perform authentification for an entire network. You don't authenticate yourself to the machine you are using, but provide your credentials to the Kerberos service. Kerberos then issues cryptographic credentials that you can present to other services as evidence of your identity. 106 | 107 | The third party is called the Key Distribution Center (KDC). The KDC consists of two parts: the Authentication Server (AS) and the Ticket Granting Server (TGS). The AS authenticates users and issues tickets for the TGS. The TGS issues tickets for network services. 108 | 109 | ![kerberos-scheme](https://intellipaat.com/blog/wp-content/uploads/2020/11/Kerberos-Architecture.jpg) 110 | 111 | ## Capabilities 112 | 113 | Capabilities are a way to give a process some of the powers of the superuser without giving it all of the powers of the superuser. Capabilities are a more fine-grained way to control the powers of a process than the traditional UNIX access control model. 114 | 115 | As an example, a process can be given the capability to bind to a network port without being given the capability to read or write files. This is useful for network services that need to bind to a low-numbered port (a port number less than 1024), which is a privileged operation. 116 | 117 | ## Linux namespaces 118 | 119 | Linux namespaces are a way to create isolated environments on a Linux system. Each namespace has its own view of the system, including its own process tree, network interfaces, and filesystem. Namespaces are used by containers to create isolated environments for running applications. 120 | 121 | As an example, a container can be given its own network namespace, so that it has its own network interfaces and IP addresses. This allows you to run multiple containers on the same system without them interfering with each other. 122 | 123 | ## Modern access control 124 | 125 | In 2001, the National Security Agency (NSA) released a new access control model called SELinux (Security-Enhanced Linux). SELinux is a mandatory access control (MAC) system that is more fine-grained than the traditional UNIX access control model. 126 | 127 | Instead of adopting SELinux or another, alternative system, the kernel maintainers developed the Linux Security Modules (LSM) framework, a kernel-level interface that allows access control systems to integrate themselves as loadable kernel modules. 128 | 129 | ## Mandatory Access Control (MAC) 130 | 131 | Mandatory Access Control (MAC) is a type of access control that is more fine-grained than the traditional UNIX access control model which is discretionary access control (DAC). In a MAC system, access control decisions are made by a central authority, rather than by the owner of the object. 132 | 133 | For example, in a MAC system, the central authority might decide that a process can read a file, but not write to it. In a DAC system, the owner of the file decides who can read and write to it. 134 | 135 | ## Role-Based Access Control (RBAC) 136 | 137 | Role-Based Access Control (RBAC) is a type of access control that is more fine-grained than the traditional UNIX access control model. In a RBAC system, access control decisions are based on the roles that users have, rather than on the users themselves. 138 | 139 | Roles are similar to groups in the traditional UNIX access control model, but they are more fine-grained. For example, in a RBAC system, a user might have the role of "administrator," which allows them to perform administrative tasks, and the role of "user," which allows them to perform normal tasks. 140 | 141 | ## SELinux: Security-Enhanced Linux 142 | 143 | SELinux is one of the oldest Linux MAC implementations. It was developed by the National Security Agency (NSA) and is now maintained by the open source community. 144 | 145 | SELinux can be used to enforce a wide variety of security policies, including role-based access control (RBAC), type enforcement (TE), and multi-level security (MLS). SELinux is a complex system that requires careful configuration to work properly. 146 | 147 | ## AppArmor 148 | 149 | AppArmor is a MAC system that is similar to SELinux. It was developed by Novell, and is now maintained by the open source community. AppArmor is designed to be easier to use than SELinux, and is often used on desktop systems. 150 | 151 | -------------------------------------------------------------------------------- /booting-and-system-management-daemons/data/boot-process.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/abdoufermat5/unix-and-linux-sysadmin-notes/10b44b43dff4235fd3e4e9755d6f60361093b1e9/booting-and-system-management-daemons/data/boot-process.png -------------------------------------------------------------------------------- /booting-and-system-management-daemons/data/subcommand-systemd.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/abdoufermat5/unix-and-linux-sysadmin-notes/10b44b43dff4235fd3e4e9755d6f60361093b1e9/booting-and-system-management-daemons/data/subcommand-systemd.png -------------------------------------------------------------------------------- /booting-and-system-management-daemons/readme.md: -------------------------------------------------------------------------------- 1 | # Chapter 2: Booting and System Management Daemons 2 | 3 | "Booting" is the standard term for "starting up a computer." It's a shortened form of the word "bootstrapping," which derives from the notion that the computer has to **"pull itself up by its own bootstraps."** 4 | 5 | The boot process consits of a few broadly defined tasks: 6 | 7 | - Finding, loading, and running bootstrapping code. 8 | - Finding, loading, and running the operating system kernel. 9 | - Running startup scripts and system daemons. 10 | - Maintaining process hygiene and managing system state transitions. 11 | 12 | ## Boot process overview 13 | 14 | Most Linux distributions now use a system manager daemon called **systemd** instead of the traditional UNIX **init**. **systemd** streamlines the boot process by adding dependency management; support for concurrent startup processes, and a comprehensive approach to logging, among other features. 15 | 16 | During the bootstrapping, the kernel is loaded into memory and begins to execute. 17 | 18 | ![Boot process](./data/boot-process.png) 19 | 20 | Before the system is fully booted, filesystems must be checked and mounted and system daemon started. These procedures are managed by a series of shell scripts (sometimes called **init scripts**) or unit files that are run in sequence by **init** or parsed by **systemd**. 21 | 22 | ## BIOS vs UEFI 23 | 24 | The **Basic Input/Output System (BIOS)** and **Unified Extensible Firmware Interface (UEFI)** are two different firmware interfaces that are used to start up a computer. 25 | 26 | In a nutshell: 27 | 28 | - **BIOS :** 29 | - Uses MBR partitioning scheme. 30 | - 16-bit processor mode. 31 | - Limited to text-based interface. 32 | - Simple and less extensible. 33 | - Lacks advanced security features. 34 | 35 | - **UEFI :** 36 | - Supports both MBR and GPT partitioning schemes. 37 | - 32-bit and 64-bit processor modes. 38 | - Graphical interface with mouse support. 39 | - Modular and extensible design. 40 | - Includes advanced security features like Secure Boot. 41 | 42 | UEFI is the more modern and feature-rich firmware interface, offering improved compatibility, security, and extensibility compared to the older BIOS. 43 | 44 | 45 | You can modify the boot order on a running system using the **efibootmgr** command. This command is used to modify the UEFI boot manager. 46 | 47 | ```bash 48 | $ efibootmgr -v 49 | 50 | BootCurrent: 0004 51 | BootOrder: 0004,0000,0001,0002,0003 52 | Boot0000* Windows Boot Manager HD(1,GPT,3e0e3e3e-3e3e-3e3e-3e3e-3e3e3e3e3e3e,0x800,0x82000)/File(\EFI\MICROSOFT\BOOT\BOOTMGFW.EFI)WINDOWS.........x...B.C.D.O.B.J.E.C.T.=.{.9.d.e.a.8.6.2.c.-.5.c.d.d.-.4.e.7.0.-.a.c.c.1.-.f.3.2.b.3.4.4.d. 53 | Boot0001* UEFI: Built-in EFI Shell VenMedia(5023b95c-db26-429b-a648-bd47664c8012)..BO 54 | Boot0002* UEFI: SanDisk Cruzer Glide 1.26, Partition 1 PciRoot(0x0)/Pci(0x14,0x0)/USB(1,0)/HD(1,MBR,0x0,0x800,0x2000)..BO 55 | Boot0003* UEFI: SanDisk Cruzer Glide 1.26, Partition 2 PciRoot(0x0)/Pci(0x14,0x0)/USB(1,0)/HD(2,MBR,0x0,0x800,0x2000)..BO 56 | Boot0004* ubuntu HD(1,GPT,3e0e3e3e-3e3e-3e3e-3e3e-3e3e3e3e3e3e,0x800,0x82000)/File(\EFI\UBUNTU\SHIMX64.EFI) 57 | 58 | $ efibootmgr -o 0000,0004 # Change the boot order to boot Windows first 59 | ``` 60 | 61 | On systems (typically those with **systemd**) that allow write access by default, **rm -rf /** can be enough to permanently destroy the system at the firmware level; in addition to removing files, **rm** also removes variables and other UEFI information accessible through **/sys**. 62 | 63 | ## GRUB 64 | 65 | The **Grand Unified Bootloader (GRUB)** is a popular boot loader that is used to boot most Linux distributions. It is a flexible and powerful boot loader that can boot multiple operating systems, load Linux kernel modules, and provide a command-line interface for troubleshooting. 66 | 67 | - Table of some grub commands: 68 | 69 | | Command | Description | 70 | | -------- | ---------------------------------------------------- | 71 | | `boot` | Boots the system from the specified kernel image. | 72 | | `help` | Displays a list of available commands. | 73 | | `linux` | Loads the specified Linux kernel. | 74 | | `reboot` | Reboots the system. | 75 | | `search` | Searches devices by file, filesystem label, or UUID. | 76 | | `usb` | Tests the USB driver. | 77 | 78 | ## System management daemons 79 | 80 | Once the kernel has been loaded and has completed its initlization process, it creates a complement of "spontaneous" processes in user space. They are called spontaneous processes because the kernel starts them autonomously--in the normal course of events, new processes are created only at the behest of existing processes. 81 | 82 | Most of the spontaneous processes are really part of the kernel implementation. They don’t necessarily correspond to programs in the filesystem. They’re not configurable, and they don’t require administrative attention. You can recognize them in ps listings by their low process IDs (PIDs) and by the fact that they’re in square brackets (e.g., [kthreadd]). 83 | 84 | The exception to this pattern is the system management daemon. It has process ID 1 and usually runs under the name **init**. The system gives init a couple of special privileges, but for the most part it’s just a user-level program like any other daemon. 85 | 86 | To serve this goal, init maintains a notion of the mode in which the system should be operating. Some commonly defined modes: 87 | 88 | - **Single-user mode** : The system is in a minimal state, with only the root filesystem mounted and only the most essential system daemons running. This mode is used for system maintenance and recovery. 89 | - **Multi-user mode** : The system is fully operational, with all filesystems mounted and all system daemons running. This is the normal mode of operation. 90 | - **Server mode** : Similar to multi-user mode, but with no GUI. 91 | 92 | The **systemd** daemon is the most widely used system manager daemon in modern Linux distributions. It is a replacement for the traditional **init** daemon and provides a number of features that make it more powerful and flexible than **init**. 93 | 94 | ## systemd 95 | 96 | Systemd is not a single daemon but a collection of programs, daemons, libraries, technologies, and kernel components. 97 | 98 | **Units and unit files** : An entity that is managed by systemd is called a unit. 99 | More specifically, a unit can be "a service, a socket, a device, a mount point, an automount point, a swap file or partition, a start-up target, a watched file system path, a timer controlled and supervised by systemd, a resource management slice or a group of externally created processes, or a wormhole into an alternate universe(haha!!)." 100 | 101 | Within systemd, the behavior of each unit is defined and configured by a unit file. In the case of a service, the unit file specifies the locationof the executable file for the daemon, tells **systemd** how to start and stop the service, and identifies any other units that the service depends on. 102 | 103 | Example of **rsync** service unit file: 104 | 105 | ```bash 106 | [Unit] 107 | Description=fast remote file copy program daemon 108 | ConditionPathExists=/etc/rsyncd.conf 109 | 110 | [Service] 111 | ExecStart=/usr/bin/rsync --daemon --no-detach 112 | 113 | [Install] 114 | WantedBy=multi-user.target 115 | ``` 116 | 117 | Unit files can live in several different places. **/usr/lib/systemd/system** is the main place where packages deposit their unit files during installation; on some systems, the path is **/lib/systemd/system** instead. The contents of this directory are considered stock, so you shouldn’t modify them. Your local unit files and customizations can go in **/etc/systemd/system**. 118 | There’s also a unit directory in **/run/systemd/system** that’s a scratch area for transient units. 119 | 120 | Service unit have a **.service** extension, socket units have a **.socket** extension, and so on. 121 | 122 | **systemctl** is the primary tool for managing systemd. It can be used to start, stop, enable, disable, and check the status of units. It can also be used to list units and their dependencies, and to show the logs of units. 123 | 124 | systemctl is an all-purpose command for investigating the status of systemd and making changes to its configuration. As with Git and several other complex software suites, systemctl’s first argument is typically a subcommand that sets the general agenda, and subsequent arguments are specific to that particular subcommand. The subcommands could be top-level commands in their own right, but for consistency and clarity, they’re bundled into the systemctl omnibus. 125 | 126 | **Examples:** 127 | 128 | ```bash 129 | # show all loaded and active services, sockets, targets, mounts, and devices 130 | $ systemctl list-units 131 | 132 | # show all loaded and active services 133 | $ systemctl list-units --type=service 134 | 135 | # show all installed unit files 136 | $ systemctl list-unit-files --type=socket 137 | 138 | ``` 139 | 140 | ![systemctl](./data/subcommand-systemd.png) 141 | 142 | The unit file statuses are: 143 | 144 | - **bad** : The unit file is bad or broken. 145 | - **disabled** : The unit file is installed but not configured to start autonomously. 146 | - **enabled** : The unit file is installed and configured to start autonomously. 147 | - **masked** : Banished from the **systemd** world from a logical perspective. 148 | - **static** : Depended upon by another unit; has no install requirements. 149 | - **indirect** : Disabled, but has peers in Also clauses that may be enabled. 150 | - **linked** : A symbolic link to another unit file. 151 | 152 | 153 | Unit files can declare their relationships to other units in a variety of ways. The most common is to specify a **WantedBy** or **RequiredBy** directive in the **[Install]** section of the unit file. These directives specify the target units that the unit should be started or stopped with. 154 | 155 | ## Rebooting and shutting down 156 | 157 | The **halt** command performs the essential duties required for shutting down the system. **halt** logs the shutdown, kill nonessential processes, flushes cached filesystem blocks to disk, and then halts the kernel. 158 | On most systems, **halt -p** powers down the system as a final flourish. 159 | 160 | **reboot** is essentially identical to halt, but it causes the machine to reboot instead of halting. 161 | 162 | The shutdown command is a layer over halt and reboot that provides for scheduled shutdowns and ominous warnings to logged-in users. It dates back to the days of time-sharing systems and is now largely obsolete. shutdown does nothing of technical value beyond halt or reboot, so feel free to ignore it if you don’t have multiuser systems -------------------------------------------------------------------------------- /cloud-computing/data/cloud-providers.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/abdoufermat5/unix-and-linux-sysadmin-notes/10b44b43dff4235fd3e4e9755d6f60361093b1e9/cloud-computing/data/cloud-providers.png -------------------------------------------------------------------------------- /cloud-computing/readme.md: -------------------------------------------------------------------------------- 1 | # Chapter 9: Cloud Computing 2 | 3 | ![Cloud Computing](https://ostechnix.com/wp-content/uploads/2021/12/Cloud-Computing.png) 4 | 5 | Cloud computing is the on-demand delivery of IT resources over the Internet with pay-as-you-go pricing. Instead of buying, owning, and maintaining physical data centers and servers, you can access technology services, such as computing power, storage, and databases, on an as-needed basis from a cloud provider like Amazon Web Services (AWS), Microsoft Azure, and Google Cloud. 6 | 7 | ## The cloud in context 8 | 9 | Cloud providers create technically advanced infrastructure that most businesses cannot hope to match. The cost of running distributed computing services is much lower for a cloud provider than for a typical business with a small data center. 10 | 11 | Cloud systems can be programmatically requested and released. This means that you can create and destroy resources programmatically, which is a powerful feature for automation and scaling. 12 | 13 | In the cloud, you are no longer restricted by slow procurement or provisioning processes, and nealy everything can be automated. 14 | 15 | Still, a certain mental leap is required when you don't control your own hardware. One industry metaphor captures the sentiment neatly: *servers should be treated as cattle, not pets*. A pet server is lovingly cared for, and when it gets sick, you nurse it back to health. A cattle server is one of many, and when it gets sick, you replace it with another one. 16 | 17 | Despite all its advantages, the cloud is not a panacea for quickly reducing costs or improving performance. Directly migrating an existing entreprise application from a data center to a cloud provides (a so-called **"lift and shift"**) is unlikely to be successful without careful planning. 18 | 19 | ## Cloud platform choices 20 | 21 | ![cloud-providers](./data/cloud-providers.png) 22 | 23 | ### Public, private, and hybrid clouds 24 | 25 | - **Public cloud**: A public cloud is a cloud service offered by a third-party provider, such as Amazon Web Services (AWS), Google Cloud, or Microsoft Azure. Public clouds are owned and operated by a third-party cloud service provider, which deliver their computing resources like servers and storage over the Internet. With a public cloud, all hardware, software, and other supporting infrastructure is owned and managed by the cloud provider. You access these services and manage your account using a web browser. 26 | - **Private cloud**: A private cloud refers to cloud computing resources used exclusively by a single business or organization. A private cloud can be physically located on the company's on-site datacenter. Some companies also pay third-party service providers to host their private cloud. A private cloud is one in which the services and infrastructure are maintained on a private network. 27 | 28 | OpenStack is the leading open source system used to create private clouds. It receives financial and engineering support from many of the world's largest companies, including IBM, Red Hat, and Rackspace. 29 | 30 | - **Hybrid cloud**: A combination of public and private clouds is called a hybrid cloud. Hybrids can be useful when an enterprise is first migrating from local servers to a public cloud, for adding temporary capacity to handle peak loads, and for a variety of other organization-specific scenarios. Administrators beware: operating two distinct cloud presences in tandem increases complexity more than proportionally. 31 | 32 | ### Cloud service fundamentals 33 | 34 | Cloud services are loosely grouped into three categories: 35 | 36 | - **Infrastructure as a Service (IaaS)**, in which users request raw compute, memory, network, and storage resources. These are typically delivered in the form of virtual private servers, aka VPSs. Under IaaS, users are responsible for managing everything above the hardware: operating systems, networking, storage systems, and their own software. 37 | - **Platform as a Service (PaaS)**, in which developers submit their custom applications packaged in a format specified by the vendor. The vendor then runs the code on the user’s behalf. In this model, users are responsible for their own code, while the vendor manages the OS and network. 38 | - **Software as a Service (SaaS)**, the broadcasr category, in which the vendor hosts and manages software and users pay some form of subscription fee for access. Users maintain neither the operating system nor the application. Almost any hosted web application (think WordPress) falls into the SaaS category. 39 | 40 | ![cloud-services](https://www.redhat.com/rhdc/managed-files/iaas-paas-saas-diagram5.1-1638x1046.png) 41 | 42 | ### Regions and availability zones 43 | 44 | A "region" is a physical location in the world where a cloud provider has data centers. In most cases, regions are named after the territory of intended service even though the data centers themselves are more concentrated. For example, the "us-east-1" region of AWS is located in northern Virginia. 45 | 46 | Some providers also have “availability zones” (or simply “zones”) which are collections of data centers within a region. ZOnes within a region are peered through high-bandwidth, low-latency, redundant circuits, so inter-zone communication is fast, though not necessarily cheap. Availability zones are designed to be isolated from each other, so that a failure in one zone does not affect the others. 47 | 48 | Regions and zones are fundamental to building highly available network services. Depending on availability requirements, you can deploy in multiple zones and regions to minimize the impact of a failure within a datacenter or geographic area. 49 | 50 | Multiregion deployments are more complex because of the physical distance between regions and the associated higher latency. 51 | 52 | ### Virtual Private Servers 53 | 54 | The flagship service of the cloud is the virtual private server, a virtual machine that runs on the provider's hardware. VPS are sometimes called instances. 55 | 56 | Instances are created from “images,” the saved state of an operating system that contains (at minimum) a root filesystem and a boot loader. An image might also include disk volumes for additional filesystems and other custom settings. 57 | 58 | ### Networking 59 | 60 | Cloud providers let you create virtual networks with custom topologies that isolate your systems from each other and from the internet. 61 | 62 | You can make your servers accessible to the Internet by leasing publicly routable addresses from your provider (e.g. Elastic IPs on AWS)--all providers have a large pool of such addresses from which users can draw. 63 | Alternatively, servers can be given only a private RFC1918 address within the address space you selected for your network, rendering them publicly inaccessible. 64 | 65 | Systems without public addresses are not directly accessible from the Internet, even for administrative attention. You can access such hosts through a jump server or bastion host that is open to the Internet, or through a VPN that connects to your cloud network. For security, the smaller the external-facing footprint of your virtual empire, the better. 66 | 67 | ### Storage 68 | 69 | The cloud vendors bill by the amount of data you store. They are highly motivated to give you as many ways as possible to ingest your data. 70 | 71 | Here are a few of the most important ways to store data in the cloud: 72 | 73 | - **Object stores** contain collection of discrete objects (files, essentially) in a flat namespace. Object stores can accommodate a virtually unlimited amount of data with exceptionally high reliability but relatively slow performance. Examples include Amazon S3, Google Cloud Storage, and Azure Blob Storage. 74 | - **Block storage** devices are virtualized hard disks that can be attached to instances. They are faster than object stores but are limited in size and are more expensive. Examples include Amazon EBS, Google Persistent Disk, and Azure Disk Storage. 75 | - **Ephemeral storage** is local disk space on a VPS that is created from disk drives on the host server. Ephemeral storage is fast but is lost when the instance is terminated. It is useful for temporary files or caches. Examples include instance store on AWS and local SSDs on Google Cloud. 76 | 77 | ### Identity and authorization 78 | 79 | AWS is exceptionally strong in this area. Their service, called Identity and Access Management (IAM), defines not only users and groups but also roles for systems. A server can be assigned policies, for example, to allow its software to start and stop other servers, store and retrieve data in an object store, or interact with queues—all with automatic key rotation. IAM also has an API for key management to help you store secrets safely. 80 | 81 | Other cloud platforms have fewer authorization features. Unsurprisingly, Azure’s service is based on Microsoft’s Active Directory. It pairs well with sites that have an existing directory to integrate with. Google’s access control service, also called IAM, is relatively coarse-grained and incomplete in comparison with Amazon’s. 82 | 83 | ## Clouds: vps 84 | 85 | ### Amazon Web Services 86 | 87 | By default, EC2 instances in VPC subnets do not have public IP addresses attached, rendering them accessible only from other systems within the same VPC. 88 | 89 | Firewalls in EC2 are known as "security groups." If you don't specify a security group, AWS will assume the "default" group, which allows no access. To connect to the instance, adjust the security group to permit SSH from your IP address. 90 | 91 | ### GCP 92 | 93 | **gcloud**(the CLI) initializes the instance with a public and private IP address. You can use the public IP with SSH, but gcloud has a helpful wrapper to simplify SSH logins. -------------------------------------------------------------------------------- /config-management/data/a-vs-s.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/abdoufermat5/unix-and-linux-sysadmin-notes/10b44b43dff4235fd3e4e9755d6f60361093b1e9/config-management/data/a-vs-s.png -------------------------------------------------------------------------------- /config-management/data/ansible-play.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/abdoufermat5/unix-and-linux-sysadmin-notes/10b44b43dff4235fd3e4e9755d6f60361093b1e9/config-management/data/ansible-play.png -------------------------------------------------------------------------------- /config-management/data/ansible_diagram.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/abdoufermat5/unix-and-linux-sysadmin-notes/10b44b43dff4235fd3e4e9755d6f60361093b1e9/config-management/data/ansible_diagram.png -------------------------------------------------------------------------------- /config-management/data/cm-tools.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/abdoufermat5/unix-and-linux-sysadmin-notes/10b44b43dff4235fd3e4e9755d6f60361093b1e9/config-management/data/cm-tools.png -------------------------------------------------------------------------------- /config-management/data/new-client.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/abdoufermat5/unix-and-linux-sysadmin-notes/10b44b43dff4235fd3e4e9755d6f60361093b1e9/config-management/data/new-client.png -------------------------------------------------------------------------------- /config-management/data/terminology.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/abdoufermat5/unix-and-linux-sysadmin-notes/10b44b43dff4235fd3e4e9755d6f60361093b1e9/config-management/data/terminology.png -------------------------------------------------------------------------------- /config-management/training/bindings-examples.md: -------------------------------------------------------------------------------- 1 | # Bindings 2 | 3 | Example variable bindings in Ansible: 4 | 5 | ```yaml 6 | # group_vars/webservers.yml 7 | http_port: 80 8 | max_clients: 200 9 | document_root: /var/www/html 10 | 11 | # playbook.yml 12 | - hosts: webservers 13 | tasks: 14 | - name: Configure Apache 15 | template: 16 | src: apache.conf.j2 17 | dest: /etc/apache2/apache2.conf 18 | ``` 19 | 20 | Teamplate bindings: 21 | 22 | ```text 23 | # templates/apache.conf.j2 24 | Listen {{ http_port }} 25 | DocumentRoot {{ document_root }} 26 | 27 | 28 | AllowOverride All 29 | MaxClients {{ max_clients }} 30 | 31 | ``` 32 | 33 | Inventory bindings: 34 | 35 | ```yaml 36 | # inventory.yml 37 | all: 38 | children: 39 | webservers: 40 | hosts: 41 | web1.example.com: 42 | http_port: 8080 43 | web2.example.com: 44 | http_port: 80 45 | dbservers: 46 | hosts: 47 | db1.example.com: 48 | mysql_port: 3306 49 | ``` 50 | 51 | Host and Group Variable Precedence: 52 | 53 | ```yaml 54 | # host_vars/web1.example.com.yml 55 | http_port: 8080 56 | ssl_enabled: true 57 | 58 | # group_vars/all.yml 59 | http_port: 80 60 | ssl_enabled: false 61 | ``` 62 | 63 | Role-based Bindings: 64 | 65 | ```yaml 66 | # roles/webserver/defaults/main.yml 67 | http_port: 80 68 | ssl_enabled: false 69 | 70 | # roles/webserver/tasks/main.yml 71 | - name: Install Apache 72 | apt: 73 | name: apache2 74 | state: present 75 | 76 | - name: Configure Apache 77 | template: 78 | src: apache.conf.j2 79 | dest: /etc/apache2/apache2.conf 80 | vars: 81 | custom_port: "{{ http_port }}" 82 | ``` 83 | 84 | Conditional bindings: 85 | 86 | ```yaml 87 | # playbook.yml 88 | - hosts: all 89 | tasks: 90 | - name: Install packages based on OS 91 | package: 92 | name: "{{ item }}" 93 | state: present 94 | vars: 95 | packages: 96 | RedHat: 97 | - httpd 98 | - mod_ssl 99 | Debian: 100 | - apache2 101 | - apache2-ssl 102 | loop: "{{ packages[ansible_os_family] }}" 103 | ``` 104 | 105 | Facts as bindings: 106 | 107 | ```yaml 108 | # playbook.yml 109 | - hosts: all 110 | tasks: 111 | - name: Configure system based on available memory 112 | template: 113 | src: system.conf.j2 114 | dest: /etc/system.conf 115 | vars: 116 | max_workers: "{{ (ansible_memtotal_mb / 512) | int }}" 117 | ``` 118 | 119 | Environment-specific Bindings: 120 | 121 | ```yaml 122 | # environments/production/group_vars/all.yml 123 | environment: production 124 | log_level: warn 125 | backup_retention: 30 126 | 127 | # environments/staging/group_vars/all.yml 128 | environment: staging 129 | log_level: debug 130 | backup_retention: 7 131 | ``` -------------------------------------------------------------------------------- /containers/data/bridge-net-docker.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/abdoufermat5/unix-and-linux-sysadmin-notes/10b44b43dff4235fd3e4e9755d6f60361093b1e9/containers/data/bridge-net-docker.png -------------------------------------------------------------------------------- /containers/data/docker-archi.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/abdoufermat5/unix-and-linux-sysadmin-notes/10b44b43dff4235fd3e4e9755d6f60361093b1e9/containers/data/docker-archi.png -------------------------------------------------------------------------------- /containers/data/docker-group-issue.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/abdoufermat5/unix-and-linux-sysadmin-notes/10b44b43dff4235fd3e4e9755d6f60361093b1e9/containers/data/docker-group-issue.png -------------------------------------------------------------------------------- /containers/data/sched-arch.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/abdoufermat5/unix-and-linux-sysadmin-notes/10b44b43dff4235fd3e4e9755d6f60361093b1e9/containers/data/sched-arch.png -------------------------------------------------------------------------------- /containers/data/storage-drivers.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/abdoufermat5/unix-and-linux-sysadmin-notes/10b44b43dff4235fd3e4e9755d6f60361093b1e9/containers/data/storage-drivers.png -------------------------------------------------------------------------------- /containers/data/ufs-docker.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/abdoufermat5/unix-and-linux-sysadmin-notes/10b44b43dff4235fd3e4e9755d6f60361093b1e9/containers/data/ufs-docker.png -------------------------------------------------------------------------------- /containers/training/mesos.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/abdoufermat5/unix-and-linux-sysadmin-notes/10b44b43dff4235fd3e4e9755d6f60361093b1e9/containers/training/mesos.pdf -------------------------------------------------------------------------------- /containers/training/modern-guide-to-container-monitoring-and-orchestration.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/abdoufermat5/unix-and-linux-sysadmin-notes/10b44b43dff4235fd3e4e9755d6f60361093b1e9/containers/training/modern-guide-to-container-monitoring-and-orchestration.pdf -------------------------------------------------------------------------------- /continuous-integration-and-delivery/data/ci-cd-pipeline.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/abdoufermat5/unix-and-linux-sysadmin-notes/10b44b43dff4235fd3e4e9755d6f60361093b1e9/continuous-integration-and-delivery/data/ci-cd-pipeline.png -------------------------------------------------------------------------------- /continuous-integration-and-delivery/data/cont-based-deploy.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/abdoufermat5/unix-and-linux-sysadmin-notes/10b44b43dff4235fd3e4e9755d6f60361093b1e9/continuous-integration-and-delivery/data/cont-based-deploy.png -------------------------------------------------------------------------------- /continuous-integration-and-delivery/data/example-app.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/abdoufermat5/unix-and-linux-sysadmin-notes/10b44b43dff4235fd3e4e9755d6f60361093b1e9/continuous-integration-and-delivery/data/example-app.png -------------------------------------------------------------------------------- /continuous-integration-and-delivery/data/rc-release.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/abdoufermat5/unix-and-linux-sysadmin-notes/10b44b43dff4235fd3e4e9755d6f60361093b1e9/continuous-integration-and-delivery/data/rc-release.png -------------------------------------------------------------------------------- /continuous-integration-and-delivery/training/CI-CD-Pipelines-Guide.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/abdoufermat5/unix-and-linux-sysadmin-notes/10b44b43dff4235fd3e4e9755d6f60361093b1e9/continuous-integration-and-delivery/training/CI-CD-Pipelines-Guide.pdf -------------------------------------------------------------------------------- /dns/data/bind-statements.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/abdoufermat5/unix-and-linux-sysadmin-notes/10b44b43dff4235fd3e4e9755d6f60361093b1e9/dns/data/bind-statements.png -------------------------------------------------------------------------------- /dns/data/dns-delegation.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/abdoufermat5/unix-and-linux-sysadmin-notes/10b44b43dff4235fd3e4e9755d6f60361093b1e9/dns/data/dns-delegation.png -------------------------------------------------------------------------------- /dns/data/dns-example.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/abdoufermat5/unix-and-linux-sysadmin-notes/10b44b43dff4235fd3e4e9755d6f60361093b1e9/dns/data/dns-example.png -------------------------------------------------------------------------------- /dns/data/dns-record-types.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/abdoufermat5/unix-and-linux-sysadmin-notes/10b44b43dff4235fd3e4e9755d6f60361093b1e9/dns/data/dns-record-types.png -------------------------------------------------------------------------------- /dns/data/dns-zone-tree.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/abdoufermat5/unix-and-linux-sysadmin-notes/10b44b43dff4235fd3e4e9755d6f60361093b1e9/dns/data/dns-zone-tree.png -------------------------------------------------------------------------------- /dns/data/ns-taxonomy.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/abdoufermat5/unix-and-linux-sysadmin-notes/10b44b43dff4235fd3e4e9755d6f60361093b1e9/dns/data/ns-taxonomy.png -------------------------------------------------------------------------------- /dns/data/sec-feat-bind.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/abdoufermat5/unix-and-linux-sysadmin-notes/10b44b43dff4235fd3e4e9755d6f60361093b1e9/dns/data/sec-feat-bind.png -------------------------------------------------------------------------------- /dns/data/signed-zone.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/abdoufermat5/unix-and-linux-sysadmin-notes/10b44b43dff4235fd3e4e9755d6f60361093b1e9/dns/data/signed-zone.png -------------------------------------------------------------------------------- /dns/data/srv-atrust.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/abdoufermat5/unix-and-linux-sysadmin-notes/10b44b43dff4235fd3e4e9755d6f60361093b1e9/dns/data/srv-atrust.png -------------------------------------------------------------------------------- /dns/training/dns-e2e-transfer.md: -------------------------------------------------------------------------------- 1 | # DNS Hashing and Digital Signatures 2 | 3 | **The Problem:** 4 | 5 | DNS zones can contain a large amount of data (think of all the records for a domain like `google.com`). Encrypting this entire dataset with public-key cryptography (like RSA) would be computationally expensive and slow down DNS responses significantly. Remember, DNS is meant to be fast to provide quick website lookups. 6 | 7 | **The Solution: Hashing and Digital Signatures** 8 | 9 | 1. **Hashing:** Instead of encrypting the entire zone, DNSSEC uses a cryptographic hash function (like SHA-256). This function takes the zone data and creates a unique, fixed-length "fingerprint" called a hash. This hash is much smaller than the original data. 10 | 11 | 2. **Signing the Hash:** The zone's private key (kept secret) is used to encrypt the hash. This encrypted hash is called a **digital signature**. Think of it like a wax seal on a letter, proving authenticity. 12 | 13 | 3. **RRSIG Records:** These digital signatures are stored in special DNS records called RRSIG (Resource Record Signature) records. These records are included alongside the regular DNS records in the zone file. 14 | 15 | **Verification Process:** 16 | 17 | When a client (like your computer) receives a DNS response, it can verify the authenticity and integrity of the data: 18 | 19 | 1. **Retrieving Public Key:** The client obtains the zone's public key (which is publicly available) via a DNSKEY record. 20 | 2. **Decrypting the Signature:** Using the public key, the client decrypts the RRSIG signature. This reveals the original hash of the zone data. 21 | 3. **Hashing the Data:** The client applies the same hash function to the received DNS data, generating its own hash. 22 | 4. **Comparison:** The client compares the hash it calculated with the hash decrypted from the RRSIG record. 23 | * **If they match:** The data is authentic and hasn't been tampered with. 24 | * **If they don't match:** The data is either corrupted or has been maliciously altered. 25 | 26 | **Benefits:** 27 | 28 | * **Authenticity:** DNSSEC ensures that the DNS data you receive actually comes from the authoritative source and hasn't been spoofed by an attacker. 29 | * **Data Integrity:** It verifies that the DNS data hasn't been modified in transit, preventing attacks like cache poisoning. 30 | * **Security:** By protecting the integrity of DNS data, DNSSEC helps prevent various attacks like DNS hijacking and man-in-the-middle attacks. 31 | 32 | **Key Points:** 33 | 34 | * DNSSEC doesn't encrypt the actual data, only the hash of the data. 35 | * The security of DNSSEC relies on the secrecy of the zone's private key. 36 | * DNSSEC requires both the DNS servers and DNS resolvers (clients) to support it. 37 | 38 | -------------------------------------------------------------------------------- /dns/training/dns-trace.py: -------------------------------------------------------------------------------- 1 | import subprocess 2 | 3 | def run_command(command): 4 | """Run a system command and return its output.""" 5 | result = subprocess.run(command, stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True) 6 | return result.stdout if result.returncode == 0 else result.stderr 7 | 8 | def parse_dig_output(output): 9 | """Parse and format the output of the dig command.""" 10 | lines = output.splitlines() 11 | formatted_output = "DNS Resolution Information:\n" 12 | 13 | for line in lines: 14 | if line.startswith(";; QUESTION SECTION:"): 15 | formatted_output += "\nQuestion Section:\n" 16 | elif line.startswith(";; ANSWER SECTION:"): 17 | formatted_output += "\nAnswer Section:\n" 18 | elif line.startswith(";; AUTHORITY SECTION:"): 19 | formatted_output += "\nAuthority Section:\n" 20 | elif line.startswith(";; ADDITIONAL SECTION:"): 21 | formatted_output += "\nAdditional Section:\n" 22 | 23 | if line.startswith(";"): 24 | continue 25 | 26 | formatted_output += line + "\n" 27 | 28 | return formatted_output 29 | 30 | def parse_traceroute_output(output): 31 | """Parse and format the output of the traceroute command.""" 32 | lines = output.splitlines() 33 | formatted_output = "Traceroute Information:\n" 34 | 35 | for line in lines: 36 | formatted_output += line + "\n" 37 | 38 | return formatted_output 39 | 40 | def main(domain): 41 | """Main function to execute dig and traceroute commands and display the results.""" 42 | print(f"Resolving DNS and tracing route for: {domain}\n") 43 | 44 | # Run dig command 45 | dig_command = ["dig", domain] 46 | dig_output = run_command(dig_command) 47 | formatted_dig_output = parse_dig_output(dig_output) 48 | print(formatted_dig_output) 49 | 50 | # Run traceroute command 51 | traceroute_command = ["traceroute", domain] 52 | traceroute_output = run_command(traceroute_command) 53 | formatted_traceroute_output = parse_traceroute_output(traceroute_output) 54 | print(formatted_traceroute_output) 55 | 56 | if __name__ == "__main__": 57 | domain = input("Enter the domain you want to resolve: ") 58 | main(domain) 59 | -------------------------------------------------------------------------------- /dns/training/rec-vs-norec.md: -------------------------------------------------------------------------------- 1 | # Recursive queries vs non-recursive queries 2 | 3 | **Non-Recursive Query:** 4 | 5 | 1. **Using `dig`:** 6 | ```bash 7 | dig @ +norec 8 | ``` 9 | * Replace `` with the IP address of the DNS server you want to query (e.g., `8.8.8.8` for Google DNS). 10 | * Replace `` with the domain you want to look up (e.g., `www.example.com`). 11 | * The `+norec` option tells `dig` to perform a non-recursive query. 12 | 13 | 2. **Using `nslookup`:** 14 | ```bash 15 | nslookup -norec 16 | ``` 17 | 18 | **Example:** 19 | 20 | ```bash 21 | dig @8.8.8.8 www.example.com +norec 22 | ``` 23 | 24 | **Recursive Query:** 25 | 26 | 1. **Using `dig`:** 27 | ```bash 28 | dig 29 | ``` 30 | * By default, `dig` will perform a recursive query if no DNS server is specified. It will use the DNS servers configured in your system's `/etc/resolv.conf` file. 31 | 32 | 2. **Using `nslookup`:** 33 | ```bash 34 | nslookup 35 | ``` 36 | * Similar to `dig`, `nslookup` will perform a recursive query by default. 37 | 38 | **Example:** 39 | 40 | ```bash 41 | dig www.example.com 42 | ``` 43 | 44 | **Explanation:** 45 | 46 | * **Non-Recursive:** Your query is sent directly to the specified DNS server. If that server has the information (either cached or authoritative), it will respond with the answer. If not, it will likely return an error or a referral. 47 | * **Recursive:** Your query is sent to your local DNS server (usually your ISP's server). If that server doesn't have the answer, it will recursively query other servers until it finds the authoritative one and returns the answer to you. 48 | 49 | **Additional Notes:** 50 | 51 | * You might need to install `dig` on your system if it's not already available. It's usually included in `dnsutils` or similar packages. 52 | * The output of these commands will show the response from the DNS server, including the IP address(es) associated with the domain you queried. 53 | -------------------------------------------------------------------------------- /dns/training/registrar-regsitry.md: -------------------------------------------------------------------------------- 1 | # Registrar and registry 2 | 3 | **Think of it like a real estate transaction:** 4 | 5 | * **Registry (The Land Owner):** The registry is the organization that owns and manages a particular Top-Level Domain (TLD), like `.com` or `.org`. They set the rules for that TLD, like pricing, eligibility requirements, and technical specifications. Verisign, for instance, is the registry for the `.com` TLD. 6 | 7 | * **Registrar (The Real Estate Agent):** The registrar is the company you go through to register your domain name (which includes the Second-Level Domain or SLD). They are accredited by ICANN to sell and manage domain names on behalf of the registry. Think of companies like GoDaddy or Namecheap. 8 | 9 | * **Top-Level Domain (TLD - The Neighborhood):** The TLD is the extension at the end of your domain name, like `.com`, `.net`, or `.org`. They are the broadest categories of domain names. 10 | 11 | * **Second-Level Domain (SLD - Your House):** The SLD is the unique part of your domain name that you choose. In `www.example.com`, "example" is the SLD. 12 | 13 | **Analogy in action:** 14 | 15 | Imagine you want to buy a house (your domain name) in a specific neighborhood (the TLD). 16 | 17 | 1. You find a real estate agent (the registrar) who specializes in that neighborhood. 18 | 2. The real estate agent helps you find a suitable house (your SLD) within that neighborhood. 19 | 3. They work with the landowner (the registry) to complete the transaction, making you the official owner of that house within the neighborhood. 20 | 21 | **Key differences:** 22 | 23 | * **Registry:** 24 | - Owns and manages the TLD 25 | - Sets the rules and policies for the TLD 26 | - Maintains the database of all registered domain names within that TLD 27 | 28 | * **Registrar:** 29 | - Accredited to sell and manage domain names 30 | - Provides services like domain registration, renewal, and DNS management 31 | - Interacts with the registry on your behalf 32 | 33 | **Example:** 34 | 35 | If you want to register `mywebsite.com`, you would go to a registrar like GoDaddy. They would check if `mywebsite` is available in the `.com` TLD, managed by the Verisign registry. If it's available, they would register it for you, working with Verisign to make it official. -------------------------------------------------------------------------------- /dns/training/reverse-dns.md: -------------------------------------------------------------------------------- 1 | # Reverse DNS Delegation Conversation 2 | 3 | Realistic conversation between an ISP (Internet Service Provider) representative and a root DNS server administrator regarding delegation for reverse DNS. 4 | 5 | --- 6 | 7 | ### ISP Representative: John 8 | ### Root DNS Server Administrator: Alice 9 | 10 | **John (ISP Representative):** Hi Alice, thanks for taking the time to speak with me today. I'm John from ExampleNet ISP, and we need to set up reverse DNS delegation for a new block of IP addresses we've been assigned. 11 | 12 | **Alice (Root DNS Server Admin):** Hi John, no problem at all. I'd be happy to help. Could you provide the details of the IP address block that needs the reverse DNS delegation? 13 | 14 | **John:** Sure, we've been allocated the `192.168.1.0/24` block, and we need to ensure that reverse DNS lookups for this range are directed to our DNS servers. 15 | 16 | **Alice:** Got it. Just to confirm, you're looking to delegate the `1.168.192.in-addr.arpa` zone to your DNS servers. Is that correct? 17 | 18 | **John:** Yes, that's correct. We have two DNS servers that will be authoritative for this reverse DNS zone. Their names are `ns1.examplenet.com` and `ns2.examplenet.com`. 19 | 20 | **Alice:** Great. Could you provide the IP addresses of these name servers for verification? 21 | 22 | **John:** Sure. The IP address for `ns1.examplenet.com` is `203.0.113.1` and for `ns2.examplenet.com` is `203.0.113.2`. 23 | 24 | **Alice:** Thank you. I will need to update the root DNS servers and the relevant RIR (Regional Internet Registry) to delegate this zone to your DNS servers. I will create the NS records for `1.168.192.in-addr.arpa` pointing to `ns1.examplenet.com` and `ns2.examplenet.com`. 25 | 26 | **John:** That sounds perfect. Is there anything else you need from me to complete this delegation? 27 | 28 | **Alice:** Just to make sure everything is in order, please ensure that your DNS servers are configured correctly to serve the `1.168.192.in-addr.arpa` zone. Once the delegation is done, queries for reverse lookups in this range will be directed to your servers. 29 | 30 | **John:** Absolutely, our DNS team has already set up the zone file. Here’s a quick overview of our zone file for verification: 31 | 32 | ```plaintext 33 | $TTL 86400 34 | @ IN SOA ns1.examplenet.com. admin.examplenet.com. ( 35 | 2024060101 ; Serial 36 | 3600 ; Refresh 37 | 1800 ; Retry 38 | 1209600 ; Expire 39 | 86400 ) ; Minimum TTL 40 | 41 | @ IN NS ns1.examplenet.com. 42 | @ IN NS ns2.examplenet.com. 43 | 44 | 100 IN PTR host1.examplenet.com. 45 | 101 IN PTR host2.examplenet.com. 46 | ``` 47 | 48 | **Alice:** That looks good. I’ll proceed with the updates to the root DNS servers and notify the RIR to update their records. This process might take a little while for full propagation, but you should start seeing the effects soon. 49 | 50 | **John:** Thanks, Alice. I appreciate your help. Is there a way for us to check the status of the delegation? 51 | 52 | **Alice:** Yes, you can use tools like `dig` or `nslookup` to verify the delegation and PTR records. For example, you can run: 53 | ```sh 54 | dig -x 192.168.1.100 55 | ``` 56 | This will help you verify that the reverse DNS queries are correctly resolving. 57 | 58 | **John:** Perfect. We'll monitor that and make sure everything is working as expected. Thanks again for your assistance! 59 | 60 | **Alice:** You’re welcome, John. If you encounter any issues or need further assistance, feel free to reach out. Have a great day! 61 | 62 | --- 63 | 64 | Credit: This conversation is generated by chat GPT-4 from OpenAI. -------------------------------------------------------------------------------- /dns/training/rfc-1033.md: -------------------------------------------------------------------------------- 1 | # Cloud providers and RFC 1033 2 | 3 | **Scenario:** 4 | 5 | You're hosting your website (`example.com`) on a cloud provider's load balancer, which has a dynamic IP address that can change over time. You want to point your apex domain (you root domain) to this load balancer. 6 | 7 | **Traditional Approach (Doesn't Work):** 8 | 9 | You can't simply create a CNAME record for the apex domain pointing to the load balancer's hostname (e.g., `lb-123456789.us-east-1.elb.amazonaws.com`). This violates RFC 1033, which prohibits CNAMEs at the apex. 10 | 11 | **Cloud Provider Solutions:** 12 | 13 | 1. **ALIAS Record (AWS Route 53):** 14 | 15 | * In Route 53, you create an ALIAS record: 16 | 17 | ``` 18 | Name: example.com. 19 | Type: ALIAS 20 | Alias: lb-123456789.us-east-1.elb.amazonaws.com. 21 | ``` 22 | 23 | * Externally: When a DNS resolver queries for `example.com`, it receives an A record pointing to the current IP address of the load balancer. 24 | 25 | * Internally: Route 53 manages the resolution by keeping the A record updated whenever the load balancer's IP changes. 26 | 27 | 2. **CNAME Flattening (Cloudflare):** 28 | 29 | * In Cloudflare, you create a CNAME record: 30 | 31 | ``` 32 | Name: example.com. 33 | Type: CNAME 34 | Target: lb-123456789.us-east-1.elb.amazonaws.com. 35 | ``` 36 | 37 | * Externally: Cloudflare intercepts the query for `example.com`, resolves the CNAME to the load balancer's IP, and returns that A record to the resolver. 38 | 39 | * Internally: Cloudflare handles the CNAME resolution and keeps track of the load balancer's IP changes. 40 | 41 | **Benefits:** 42 | 43 | * **Compliance:** Both solutions comply with RFC 1033 by presenting A records to the outside world. 44 | * **Ease of Use:** You can configure the apex record like a CNAME, but the provider handles the underlying A record management. 45 | * **Dynamic Updates:** The provider automatically keeps the A record in sync with the actual target's IP address, ensuring your website remains accessible even if the IP changes. 46 | 47 | **Key Point:** The external user never sees the CNAME record. They only receive the final, resolved A record pointing to the correct IP address, ensuring seamless access to your website. 48 | -------------------------------------------------------------------------------- /docker-compose.yml: -------------------------------------------------------------------------------- 1 | version: '3.8' 2 | 3 | services: 4 | centos: 5 | build: 6 | context: . 7 | dockerfile: ./lab-volumes/Dockerfile.centos 8 | container_name: lab-centos 9 | tty: true 10 | stdin_open: true 11 | volumes: 12 | - ./lab-volumes/centos:/root:rw 13 | networks: 14 | lab-net: 15 | ipv4_address: 192.168.50.2 16 | 17 | debian: 18 | build: 19 | context: . 20 | dockerfile: ./lab-volumes/Dockerfile.debian 21 | container_name: lab-debian 22 | tty: true 23 | stdin_open: true 24 | volumes: 25 | - ./lab-volumes/debian:/root:rw 26 | networks: 27 | lab-net: 28 | ipv4_address: 192.168.50.4 29 | 30 | networks: 31 | lab-net: 32 | driver: bridge 33 | name: lab-net 34 | ipam: 35 | config: 36 | - subnet: 192.168.50.0/24 37 | gateway: 192.168.50.1 38 | -------------------------------------------------------------------------------- /drivers-and-the-kernel/data/components-device-file.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/abdoufermat5/unix-and-linux-sysadmin-notes/10b44b43dff4235fd3e4e9755d6f60361093b1e9/drivers-and-the-kernel/data/components-device-file.png -------------------------------------------------------------------------------- /drivers-and-the-kernel/data/kernel-dev-drivers.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/abdoufermat5/unix-and-linux-sysadmin-notes/10b44b43dff4235fd3e4e9755d6f60361093b1e9/drivers-and-the-kernel/data/kernel-dev-drivers.png -------------------------------------------------------------------------------- /drivers-and-the-kernel/data/udevd-match-keys.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/abdoufermat5/unix-and-linux-sysadmin-notes/10b44b43dff4235fd3e4e9755d6f60361093b1e9/drivers-and-the-kernel/data/udevd-match-keys.png -------------------------------------------------------------------------------- /drivers-and-the-kernel/readme.md: -------------------------------------------------------------------------------- 1 | # Chapter 11: Drivers and the Kernel 2 | 3 | ![kernel-dev-drivers](./data/kernel-dev-drivers.png) 4 | 5 | The kernel is the central government of a UNIX or Linux system. It’s responsible for enforcing rules, sharing resources, and providing the core services that user processes rely on. 6 | 7 | The kernel hides the details of the system's hardware underneath an abstract, high-level interface. It's akin to an API for application programmers: a well-defined interface that provides useful facilities for interacting with the system. This interface provides five basic features: 8 | 9 | - Management and abstraction of hardware devices 10 | - Processes and threads (and ways to communicate among them) 11 | - Management of memory (virtual memory and memory-space protection) 12 | - I/O facilities (filesystems, network interfaces, serial interfaces, etc.) 13 | - Housekeeping functions (startup, shutdown, timers, multitasking, etc.) 14 | 15 | Only device drivers are aware of the specific capabilities and communication protocols of the system’s hardware. User programs and the rest of the kernel are largely independent of that knowledge. 16 | 17 | For example, a filesystem on disk is very different from a network filesystem, but the kernel's VFS layer makes them look the same to user processes and to other parts of the kernel. 18 | 19 | --- 20 | 21 | ## Kernel version numbering 22 | 23 | ### Linux kernel versions 24 | 25 | You can check with `uname -r` to see what kernel a given system is running. 26 | Linux kernels are named according to the rules of so-called semantic versioning, that is, they include three components: a major version, a minor version, and a patch level. At present, there is no predictable relationship between a version number and its intended status as a stable or development kernel; kernels are blessed as stable when the developers decide that they’re stable. 27 | 28 | ## Devices and their drivers 29 | 30 | A device driver is an abstraction layer that manages the system’s interaction with a particular type of hardware so that the restof the kernel doesn't need to know its specifics. The driver translates between the hardware commands understood by the device and a stylized programming interface defined(and used by the kernel). 31 | 32 | ### Device files and device numbers 33 | 34 | In most cases, device drivers are part of the kernel; they are not user processes. However, a driver can be accessed both from within the kernel and from user space, usually through "device files" that live in **/dev** directory. 35 | 36 | Most non-network devices have one or more corresponding files in /dev. Complex servers may support hundreds of devices. By virtue of being device files, the files in /dev each have a major and minor device number associated with them. The kernel uses these numbers to map device-file references to the corresponding driver. 37 | 38 | The major device number identifies the driver with which the file is associated (in other words, the type of device). The minor device number usually identifies which particular instance of a given device type is to be addressed. The minor device number is sometimes called the unit number. 39 | 40 | ```bash 41 | $ ls -l /dev/sda 42 | brw-rw---- 1 root disk 8, 0 2024-04-05 14:50 /dev/sda 43 | ``` 44 | 45 | This example shows the first SCSI/SATA/SAS disk on a Linux system. It has a major device number of 8 and a minor device number of 0. The major device number 8 is associated with the SCSI disk driver, and the minor device number 0 is the first disk on the system. 46 | 47 | There are actually two types of device files: block and character. A block device is read or written one block (a group of bytes, usually a multiple of 512) at a time; a character device can be read or written one byte at a time. The character "b" or "c" in the first column of the ls -l output indicates whether a device file is a block or character device. 48 | 49 | It is sometimes convenient to implement an abstraction as a device driver even it controls no actual device. Such phantom devices are called pseudo-devices. For example, a user who logs in over the network is assigned a pseudo-TTY (PTY) that looks, feels, ad smells like a serial port from the perspective of higher-level software. Some pseudo-devices are used for debugging, such as `/dev/null`, which discards all data written to it, and `/dev/zero`, which returns an infinite number of zero bytes when read, or `/dev/urandom`, which returns an infinite number of random bytes when read. 50 | 51 | When a program performs an operation on a device file, the kernel intercepts the reference, looks up the appropriate function name in a table, and transfers control to the appropriate part of the driver. 52 | 53 | ### Manual creation of device files 54 | 55 | The `mknod` command can be used to create device files manually. The syntax is: 56 | 57 | ```bash 58 | $ mknod /dev/mydevice type major minor 59 | ``` 60 | 61 | where type is either `b` for block or `c` for character, and major and minor are the major and minor device numbers, respectively. 62 | 63 | ### Modern device file management 64 | 65 | The `udevd` daemon is responsible for managing device files in modern Linux systems. It creates device files dynamically as devices are discovered or added to the system. The `udev` daemon reads its configuration from the `/etc/udev` directory and from the `/lib/udev` directory. The configuration files in `/etc/udev` override those in `/lib/udev`. 66 | 67 | ![components-device-file](./data/components-device-file.png) 68 | 69 | ### Linux device management 70 | 71 | **Sysfs:** a window into the souls of devices 72 | 73 | Sysfs was added to the Linux kernel at version 2.6. It is a virtual, in-memory filesystem implemented by the kernel to provide detailed and well-organized information about the system’s available devices, their configurations, and their state. Sysfs device information is accessible both from within the kernel and from user space. 74 | 75 | Sysfs is mounted at `/sys` and is organized as a hierarchy of directories and files. Each directory represents a device or a device class, and each file contains a piece of information about the device or class. The information in sysfs is read-only and is updated by the kernel as devices are discovered, added, or removed. 76 | 77 | - Subdirectories of `/sys` : 78 | 79 | | Directory | Description | 80 | | ------------- | ------------------------------------------------------------------ | 81 | | /sys/block | Information about block devices such as hard disks | 82 | | /sys/bus | Buses known to the kernel: PCI-E, SCSI, USB, etc. | 83 | | /sys/class | A tree organized by functional types of devices | 84 | | /sys/dev | Device information split between character and block devices | 85 | | /sys/devices | An ancestrally correct representation of all discovered devices | 86 | | /sys/firmware | Interfaces to platform-specific subsystems such as ACPI | 87 | | /sys/fs | A directory for some, but not all, filesystems known to the kernel | 88 | | /sys/kernel | Kernel internals such as cache and virtual memory status | 89 | | /sys/module | Dynamic modules loaded by the kernel | 90 | | /sys/power | A few details about the system’s power state | 91 | 92 | 93 | **udevadm:** the udev administration tool 94 | 95 | The `udevadm` command queries device information, triggers events, controls the **udevd** daemon, and monitors udev and kernel events. 96 | 97 | `udevadm` expects one of six commands as its first argument: 98 | 99 | - `info` : Display information about a device 100 | For example, `udevadm info -a -p /sys/class/net/eth0` displays information about the network interface eth0. 101 | 102 | - `trigger` : Trigger the kernel to process a device event 103 | For example, `udevadm trigger --subsystem-match=block` triggers the kernel to process block device events. 104 | 105 | - `settle` : Wait for all pending udev events to be processed 106 | For example, `udevadm settle` waits for all pending udev events to be processed. 107 | 108 | - `control` : starts and stops the udev daemon or forces it to reload its rules files 109 | For example, `udevadm control --reload` forces the udev daemon to reload its rules files. 110 | 111 | - `monitor` : Monitor udev events as they occur 112 | For example, `udevadm monitor` displays udev events as they occur. 113 | 114 | - `test` : Test a single device or a single event 115 | For example, `udevadm test /sys/class/net/eth0` tests the network interface eth0. 116 | 117 | All paths in udevadm output (such as `/devices/pci0000:00/…`) are relative to `/sys`, even though they may appear to be absolute pathnames. 118 | 119 | udevd match keys: 120 | 121 | ![udevd-match-keys](./data/udevd-match-keys.png) 122 | 123 | The assignment clauses specify actions udevd should take to handle any matching events. Their format is similar to that for match clauses. 124 | 125 | THe most important assignment key is `NAME`, which indicates how `udevd` should name a new device. 126 | 127 | Here's an example configuration for a USB flash drive. Suppose we want to make the drive's device name persist across insertions and we want the drive to be mouted and unmounted automatically. 128 | 129 | ```bash 130 | $ lsusb 131 | Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub 132 | Bus 001 Device 003: ID 0408:a061 Quanta Computer, Inc. HD User Facing 133 | Bus 001 Device 004: ID 8087:0026 Intel Corp. AX201 Bluetooth 134 | Bus 001 Device 007: ID 18f8:0f97 [Maxxter] Optical Gaming Mouse [Xtrem] 135 | Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub 136 | Bus 004 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub 137 | Bus 003 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub 138 | ``` 139 | 140 | ## Linux Kernel configuration 141 | 142 | 3 methods to configure a Linux kernel: 143 | 144 | - Modifying tunalble(dynamic) kernel configuration parameters 145 | - Building a kernel from scratch (by compiling the kernel source code) 146 | - Loading new drivers and modules into a running kernel 147 | 148 | ### Tunable Linux kernel parameters 149 | 150 | The Linux kernel has many tunable parameters that can be set at runtime. These parameters are stored in the `/proc/sys` directory. The `sysctl` command is used to read and modify these parameters. 151 | 152 | For example, to display the value of the kernel parameter `vm.swappiness`, use the following command: 153 | 154 | ```bash 155 | $ sysctl vm.swappiness # (same as /proc/sys/vm/swappiness) 156 | vm.swappiness = 60 157 | ``` 158 | 159 | ### Building a kernel from scratch 160 | 161 | The Linux kernel is a monolithic kernel, which means that it is a single, large program that contains all the essential components of the operating system. The kernel is built from source code, which is available from the kernel.org website. 162 | 163 | The kernel source code is organized into directories, each of which contains the source code for a different part of the kernel. The source code is written in the C programming language and is compiled into a binary executable that can be loaded into memory and executed by the computer. 164 | 165 | **Configuring kernel options:** 166 | 167 | The kernel source code contains a file called `.config` at the root of the source tree. This file contains the configuration options that are used to build the kernel. The configuration options are stored in a format that is similar to the output of the `make menuconfig` command. 168 | 169 | You can use the decoding guide in `kernel_src_dir/Documentation/Configure.help` to find out what the various options mean. 170 | 171 | It's usually inadvisable to edit the `.config` file directly. Instead, use one of the following methods to configure the kernel: 172 | 173 | - `make menuconfig` : A text-based menu-driven configuration tool 174 | - `make xconfig` : A graphical configuration tool if you are running KDE 175 | - `make gconfig` : A graphical configuration tool if you are running GNOME 176 | - `make oldconfig` : Update an existing `.config` file with new options 177 | 178 | **Building the kernel binary:** 179 | 180 | Here's an outline of the entire process to get a finished kernel: 181 | 182 | 1. Change directory to the kernel source directory (e.g., `cd /usr/src/linux-5.10.25`) 183 | 2. Run `make xconfig`, `make gconfig`, or `make menuconfig` to configure the kernel 184 | 3. Run `make clean`. (optional) 185 | 4. Run `make` 186 | 5. Run `make modules_install` 187 | 6. Run `make install` 188 | 189 | ### Adding a linux device driver 190 | 191 | On Linux systems, device drivers are typically distributed in one of three forms: 192 | 193 | - A patch against a specific kernel version 194 | - A loadable kernel module 195 | - An installation script or package that installs the driver 196 | 197 | ## Loadable Kernel Modules (LKM) 198 | 199 | A loadable kernel module (LKM) is a piece of code that can be loaded into the Linux kernel at runtime. LKMs are used to add new functionality to the kernel without having to recompile the entire kernel. LKMs are typically used to add support for new hardware devices or to add new features to the kernel. 200 | 201 | Under Linux, you can inspect the currently loaded kernel modules with the `lsmod` command: 202 | 203 | ```bash 204 | $ lsmod 205 | Module Size Used by 206 | nls_utf8 16384 1 207 | isofs 49152 1 208 | uas 24576 0 209 | usb_storage 77824 1 uas 210 | ``` 211 | 212 | As an example of manually loading a kernel module, here’s how we would insert a module that implements sound output to usb devices: 213 | 214 | ```bash 215 | $ sudo modprobe snd-usb-audio 216 | ``` 217 | 218 | `modprobe` is a semi-automatic wrapper around a more primitive command, `insmod`. `modprobe` understands dependencies, options, and installation and removal procedures. 219 | 220 | ## Booting 221 | 222 | Here's a brief overview of the boot process on a Linux system: 223 | 224 | 1. The BIOS or UEFI firmware initializes the hardware and loads the boot loader from the boot device (usually a hard disk). 225 | 2. The boot loader loads the kernel image into memory and starts the kernel. 226 | 3. The kernel initializes the hardware, mounts the root filesystem, and starts the init process. 227 | 4. The init process starts other system processes and services. 228 | 5. The system is now fully booted and ready for use. 229 | 230 | The boot loader is responsible for loading the kernel image into memory and starting the kernel. The most common boot loaders on Linux systems are GRUB and LILO. 231 | 232 | ## Booting alternate kernels in the cloud 233 | 234 | Cloud instances boot differently from traditional hardware. Most cloud providers sidestep GRUB and use either a modified open source boot loader or some kind of scheme that avoids the use of a boot loader altogether. On AWS, the base AMI(amazon machine image) uses a boot loader called PV-GRUB, which is a patched version of GRUB that lets you specify the kernel in the **menu.lst** file. 235 | 236 | ## Kernel errors 237 | 238 | Linux has four varieties of kernel failure: soft lockups, hard lockups, kernel panics, and oopses. 239 | 240 | - **Soft lockups** are caused by a kernel process that has been running for too long without yielding the CPU to other processes. Soft lockups are not fatal and can be recovered from. During a soft lockup, the kernel is the only thing running, but it is still servicing interrupts such as those from network interfaces and keyboards. 241 | - **Hard lockups** is the same as a soft lockup, but with the additional complication that most processor interrupts go unserviced. 242 | 243 | A soft or hard lockup is almost always the result of a hardware failure, the most common culprit being bad memory. 244 | 245 | - **oops**: The Linux “oops” system is a generalization of the traditional UNIX “panic after any anomaly” approach to kernel integrity. 246 | - **Panics**: A kernel panic is a fatal error that occurs when the kernel detects an unrecoverable error. When a kernel panic occurs, the kernel halts the system and displays a message that describes the error. The system must be rebooted to recover from a kernel panic. 247 | 248 | -------------------------------------------------------------------------------- /electronic-mail/data/dsn-error-codes.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/abdoufermat5/unix-and-linux-sysadmin-notes/10b44b43dff4235fd3e4e9755d6f60361093b1e9/electronic-mail/data/dsn-error-codes.png -------------------------------------------------------------------------------- /electronic-mail/data/evi-and-david.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/abdoufermat5/unix-and-linux-sysadmin-notes/10b44b43dff4235fd3e4e9755d6f60361093b1e9/electronic-mail/data/evi-and-david.png -------------------------------------------------------------------------------- /electronic-mail/data/mail-sys-comp.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/abdoufermat5/unix-and-linux-sysadmin-notes/10b44b43dff4235fd3e4e9755d6f60361093b1e9/electronic-mail/data/mail-sys-comp.png -------------------------------------------------------------------------------- /electronic-mail/data/mta-market.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/abdoufermat5/unix-and-linux-sysadmin-notes/10b44b43dff4235fd3e4e9755d6f60361093b1e9/electronic-mail/data/mta-market.png -------------------------------------------------------------------------------- /electronic-mail/data/postfix-arch.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/abdoufermat5/unix-and-linux-sysadmin-notes/10b44b43dff4235fd3e4e9755d6f60361093b1e9/electronic-mail/data/postfix-arch.png -------------------------------------------------------------------------------- /electronic-mail/data/smtp-cmd.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/abdoufermat5/unix-and-linux-sysadmin-notes/10b44b43dff4235fd3e4e9755d6f60361093b1e9/electronic-mail/data/smtp-cmd.png -------------------------------------------------------------------------------- /ip-routing/data/B-rtb.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/abdoufermat5/unix-and-linux-sysadmin-notes/10b44b43dff4235fd3e4e9755d6f60361093b1e9/ip-routing/data/B-rtb.png -------------------------------------------------------------------------------- /ip-routing/data/R1-rtb.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/abdoufermat5/unix-and-linux-sysadmin-notes/10b44b43dff4235fd3e4e9755d6f60361093b1e9/ip-routing/data/R1-rtb.png -------------------------------------------------------------------------------- /ip-routing/data/host-rtb.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/abdoufermat5/unix-and-linux-sysadmin-notes/10b44b43dff4235fd3e4e9755d6f60361093b1e9/ip-routing/data/host-rtb.png -------------------------------------------------------------------------------- /ip-routing/data/packet-forwarding.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/abdoufermat5/unix-and-linux-sysadmin-notes/10b44b43dff4235fd3e4e9755d6f60361093b1e9/ip-routing/data/packet-forwarding.png -------------------------------------------------------------------------------- /ip-routing/readme.md: -------------------------------------------------------------------------------- 1 | # Chapter 15: IP Routing 2 | 3 | We're going deeper into routing and how IP packets are forwarded clearly. 4 | 5 | There's a difference between the process of forwarding packets and the management of the routing table that drives this process. 6 | 7 | ## Packet Forwarding 8 | 9 | ![packet-forwarding](./data/packet-forwarding.png) 10 | 11 | Router R1 connects two networks, and router R2 connects one of these nets to the outside world. A look at the routing tables for these hosts and routers lets us examine some specific packet forwarding scenarios. First, host A’s routing table: 12 | 13 | ![host-rtb](./data/host-rtb.png) 14 | 15 | Host A has the simplest routing configuration of the four machines. The first two routes describe the machine's own network interfaces in standard routing terms. These entries exist so that forwarding to directly connected networks need not be handled as a special case. eth0 is host A’s Ethernet interface, and lo is the loopback interface, a virtual interface emulated in software. Entries such as these are normally added automatically when a network interface is configured. 16 | 17 | The default route on host A forwards all packets not addressed to the loopback address or to the **199.165.145** network to the router R1, whose address on this network is **199.165.145.24**. *Gateways must be only one hop away.* 18 | 19 | Sending a packet to **199.165.146.4** (host B) will be forwarded to R1. The packet will be sent to the MAC address of R1's interface on the **199.165.145** network. R1 after checking its routing table will forward the packet to host B since R1 has a direct connection to the **199.165.146** network. 20 | 21 | Here is the routing table for R1: 22 | 23 | ![R1-rtb](./data/R1-rtb.png) 24 | 25 | In theory, you can configure host B with initial knowledge of only one gateway and rely on help from ICMP redirects to eliminate extra hops. For example, here is one possible initial configuration for host B: 26 | 27 | ![B-rtb](./data/B-rtb.png) 28 | 29 | If B sends a packet to host A (199.165.145.17), no route matches and the packet is forwarded to R2 for delivery. R2 (which, being a router, presumably has complete knowledge of the network) will forward the packet to R1, which will deliver it to host A. Since R1 and B are on the same network, R2 will send an ICMP redirect to B, informing it that it can reach host A directly. B will then update its routing table to reflect this new information. 30 | 31 | `199.165.145.1 199.165.146.1 255.255.255.255 UGHD 0 0 0 eth0` 32 | 33 | But from a security standpoint, it is better to ignore ICMP redirects and leave the routing table as is. 34 | 35 | ## Routing daemons and routing protocols 36 | 37 | Instead of having to explicitly tell every computer on every network how to reach every other computer and network, it would be nice if the computers could just cooperate and figure it all out. This is the job of routing protocols and the daemons that implement them. 38 | 39 | ### Distance Vector Routing 40 | 41 | The simplest routing protocol is distance vector routing. In this scheme, each router periodically sends its routing table to its neighbors. Each router then updates its own routing table based on the information it receives. The information sent between routers is a list of destinations and the number of hops to reach each one. The routers do not know the topology of the network, only the number of hops to reach each destination. Over time, the routers converge on a consistent view of the network. 42 | 43 | ![distance-vector](https://www.cs.umd.edu/~shankar/417-F01/Slides/chapter4a-aus/img015.gif) 44 | 45 | Here's a list of some common distance vector routing protocols: 46 | 47 | - RIP (Routing Information Protocol): RIP is a simple protocol that uses hop count as its metric. RIP is limited to 15 hops, which makes it unsuitable for large networks. RIP is a distance vector protocol. 48 | - EIGRP (Enhanced Interior Gateway Routing Protocol): EIGRP is a Cisco proprietary protocol that uses a combination of hop count and bandwidth as its metric. 49 | - BGP (Border Gateway Protocol): BGP is used to route traffic between different autonomous systems (ASes). BGP transmits the entire routing table once and then only sends updates when the routing table changes. 50 | 51 | 52 | ### Link State Routing 53 | 54 | Link state routing is a more sophisticated routing protocol. In this scheme, each router sends a list of its neighbors to all other routers. Each router then constructs a map of the network and calculates the shortest path to each destination. The routers then use this map to forward packets. Link state routing is more complex than distance vector routing, but it is more efficient and more scalable. 55 | 56 | The primary advantage that link state routing has over distance vector is the ability to converge on an operational routing solution after a catastrophe occurs. The trade-off is that link state routing requires more memory and CPU resources to operate. 57 | 58 | Each link state includes information about: 59 | 60 | - The router itself: Identifying which router is reporting the information. 61 | - The connected neighbors: Identifying the routers directly connected to this router. 62 | - The cost of each link: This could be based on factors like bandwidth, delay, or other metrics that determine how "good" or "bad" a connection is. 63 | 64 | Here's a list of some common link state routing protocols: 65 | 66 | - OSPF (Open Shortest Path First **RFC2328**): OSPF is a link state protocol that uses the Dijkstra algorithm to calculate the shortest path to each destination. 67 | - IS-IS (Intermediate System to Intermediate System): IS-IS is a link state protocol that is similar to OSPF. IS-IS is used primarily in service provider networks. 68 | 69 | --- 70 | 71 | **AS (Autonomous System)**: A collection of IP networks and routers under the control of a single organization that presents a common routing policy to the internet. 72 | 73 | --- 74 | 75 | ## Cisco IOS 76 | 77 | Cisco routers use a proprietary operating system called Cisco IOS. The IOS command-line interface is similar to a Unix shell, with a set of commands that allow you to configure the router and monitor its operation. Here are some common commands: 78 | 79 | - `show ip route`: Displays the routing table. 80 | - `ssh mycompany.router.com`: Connects to a remote router using SSH. 81 | - `configure terminal`: Enters configuration mode. 82 | 83 | -------------------------------------------------------------------------------- /lab-volumes/Dockerfile.centos: -------------------------------------------------------------------------------- 1 | # Dockerfile.centos 2 | FROM centos/systemd 3 | 4 | # Install some basic tools 5 | RUN yum update -y && yum install -y \ 6 | curl \ 7 | vim \ 8 | && yum clean all 9 | 10 | # Set the entrypoint 11 | ENTRYPOINT ["/bin/bash"] 12 | -------------------------------------------------------------------------------- /lab-volumes/Dockerfile.debian: -------------------------------------------------------------------------------- 1 | # Dockerfile.debian 2 | FROM debian:12 3 | 4 | # Install some basic tools 5 | RUN apt-get update && apt-get install -y \ 6 | curl \ 7 | vim \ 8 | && rm -rf /var/lib/apt/lists/* 9 | 10 | # Set environment variables 11 | ENV DEBIAN_FRONTEND=noninteractive 12 | 13 | # Set the entrypoint 14 | ENTRYPOINT ["/bin/bash"] 15 | -------------------------------------------------------------------------------- /lab-volumes/centos/test: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/abdoufermat5/unix-and-linux-sysadmin-notes/10b44b43dff4235fd3e4e9755d6f60361093b1e9/lab-volumes/centos/test -------------------------------------------------------------------------------- /lab-volumes/debian/test: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/abdoufermat5/unix-and-linux-sysadmin-notes/10b44b43dff4235fd3e4e9755d6f60361093b1e9/lab-volumes/debian/test -------------------------------------------------------------------------------- /logging/data/case-study-logging.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/abdoufermat5/unix-and-linux-sysadmin-notes/10b44b43dff4235fd3e4e9755d6f60361093b1e9/logging/data/case-study-logging.png -------------------------------------------------------------------------------- /logging/data/common-action.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/abdoufermat5/unix-and-linux-sysadmin-notes/10b44b43dff4235fd3e4e9755d6f60361093b1e9/logging/data/common-action.png -------------------------------------------------------------------------------- /logging/data/facility.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/abdoufermat5/unix-and-linux-sysadmin-notes/10b44b43dff4235fd3e4e9755d6f60361093b1e9/logging/data/facility.png -------------------------------------------------------------------------------- /logging/data/level-qualif.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/abdoufermat5/unix-and-linux-sysadmin-notes/10b44b43dff4235fd3e4e9755d6f60361093b1e9/logging/data/level-qualif.png -------------------------------------------------------------------------------- /logging/data/log-files.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/abdoufermat5/unix-and-linux-sysadmin-notes/10b44b43dff4235fd3e4e9755d6f60361093b1e9/logging/data/log-files.png -------------------------------------------------------------------------------- /logging/data/logrotate-options.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/abdoufermat5/unix-and-linux-sysadmin-notes/10b44b43dff4235fd3e4e9755d6f60361093b1e9/logging/data/logrotate-options.png -------------------------------------------------------------------------------- /logging/data/rsyslog-conf.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/abdoufermat5/unix-and-linux-sysadmin-notes/10b44b43dff4235fd3e4e9755d6f60361093b1e9/logging/data/rsyslog-conf.png -------------------------------------------------------------------------------- /logging/data/rsyslog-prop.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/abdoufermat5/unix-and-linux-sysadmin-notes/10b44b43dff4235fd3e4e9755d6f60361093b1e9/logging/data/rsyslog-prop.png -------------------------------------------------------------------------------- /logging/data/severity.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/abdoufermat5/unix-and-linux-sysadmin-notes/10b44b43dff4235fd3e4e9755d6f60361093b1e9/logging/data/severity.png -------------------------------------------------------------------------------- /monitoring/data/carbon-summarization.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/abdoufermat5/unix-and-linux-sysadmin-notes/10b44b43dff4235fd3e4e9755d6f60361093b1e9/monitoring/data/carbon-summarization.png -------------------------------------------------------------------------------- /monitoring/data/commercial-platforms.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/abdoufermat5/unix-and-linux-sysadmin-notes/10b44b43dff4235fd3e4e9755d6f60361093b1e9/monitoring/data/commercial-platforms.png -------------------------------------------------------------------------------- /monitoring/data/graphite.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/abdoufermat5/unix-and-linux-sysadmin-notes/10b44b43dff4235fd3e4e9755d6f60361093b1e9/monitoring/data/graphite.png -------------------------------------------------------------------------------- /monitoring/data/snmpwalk.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/abdoufermat5/unix-and-linux-sysadmin-notes/10b44b43dff4235fd3e4e9755d6f60361093b1e9/monitoring/data/snmpwalk.png -------------------------------------------------------------------------------- /monitoring/training/cpt93_configuration_chapter_010011.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/abdoufermat5/unix-and-linux-sysadmin-notes/10b44b43dff4235fd3e4e9755d6f60361093b1e9/monitoring/training/cpt93_configuration_chapter_010011.pdf -------------------------------------------------------------------------------- /monitoring/training/net-snmp.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/abdoufermat5/unix-and-linux-sysadmin-notes/10b44b43dff4235fd3e4e9755d6f60361093b1e9/monitoring/training/net-snmp.pdf -------------------------------------------------------------------------------- /network-file-system/data/client-specs.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/abdoufermat5/unix-and-linux-sysadmin-notes/10b44b43dff4235fd3e4e9755d6f60361093b1e9/network-file-system/data/client-specs.png -------------------------------------------------------------------------------- /network-file-system/data/nfs-joke.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/abdoufermat5/unix-and-linux-sysadmin-notes/10b44b43dff4235fd3e4e9755d6f60361093b1e9/network-file-system/data/nfs-joke.jpg -------------------------------------------------------------------------------- /network-file-system/data/nfs-versions.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/abdoufermat5/unix-and-linux-sysadmin-notes/10b44b43dff4235fd3e4e9755d6f60361093b1e9/network-file-system/data/nfs-versions.png -------------------------------------------------------------------------------- /network-file-system/training/aws-efs.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/abdoufermat5/unix-and-linux-sysadmin-notes/10b44b43dff4235fd3e4e9755d6f60361093b1e9/network-file-system/training/aws-efs.pdf -------------------------------------------------------------------------------- /network-file-system/training/root-squash.md: -------------------------------------------------------------------------------- 1 | # Root-squash, no-root-squash, all-squash 2 | 3 | Let's say we have an NFS server (192.168.1.10) sharing a directory `/shared_data` with a client (192.168.1.20). 4 | 5 | 1. **root_squash** (Default behavior): 6 | ```bash 7 | # In /etc/exports on NFS server 8 | /shared_data 192.168.1.20(rw,root_squash) 9 | ``` 10 | With root_squash: 11 | - If root user (UID 0) on the client accesses the share → mapped to anonymous user (typically nobody:nogroup, UID/GID 65534) 12 | - Regular users retain their original UIDs/GIDs 13 | 14 | Example: 15 | ```bash 16 | # On client as root (UID 0) 17 | touch /mnt/shared_data/file1 # Created as nobody:nogroup 18 | ls -l /mnt/shared_data/file1 # Shows owner as nobody:nogroup 19 | 20 | # On client as user bob (UID 1000) 21 | touch /mnt/shared_data/file2 # Created as bob:bob 22 | ls -l /mnt/shared_data/file2 # Shows owner as bob:bob 23 | ``` 24 | 25 | 2. **no_root_squash**: 26 | ```bash 27 | # In /etc/exports on NFS server 28 | /shared_data 192.168.1.20(rw,no_root_squash) 29 | ``` 30 | With no_root_squash: 31 | - Root user on client retains root privileges on shared directory 32 | - Regular users retain their original UIDs/GIDs 33 | 34 | Example: 35 | ```bash 36 | # On client as root (UID 0) 37 | touch /mnt/shared_data/file1 # Created as root:root 38 | ls -l /mnt/shared_data/file1 # Shows owner as root:root 39 | 40 | # On client as user bob (UID 1000) 41 | touch /mnt/shared_data/file2 # Created as bob:bob 42 | ls -l /mnt/shared_data/file2 # Shows owner as bob:bob 43 | ``` 44 | 45 | 3. **all_squash**: 46 | ```bash 47 | # In /etc/exports on NFS server 48 | /shared_data 192.168.1.20(rw,all_squash) 49 | ``` 50 | With all_squash: 51 | - ALL users (root and regular) are mapped to anonymous user 52 | - Useful for public shares where you don't want to maintain UID/GID mapping 53 | 54 | Example: 55 | ```bash 56 | # On client as root (UID 0) 57 | touch /mnt/shared_data/file1 # Created as nobody:nogroup 58 | ls -l /mnt/shared_data/file1 # Shows owner as nobody:nogroup 59 | 60 | # On client as user bob (UID 1000) 61 | touch /mnt/shared_data/file2 # Created as nobody:nogroup 62 | ls -l /mnt/shared_data/file2 # Shows owner as nobody:nogroup 63 | ``` 64 | 65 | Common use cases: 66 | 1. **root_squash**: Most common, default security measure 67 | - Use when you want normal users to retain their identities but prevent root access 68 | - Good for shared development directories 69 | 70 | 2. **no_root_squash**: Use with caution! 71 | - Backup systems that need root access 72 | - System maintenance tasks 73 | - Automated administrative scripts 74 | 75 | 3. **all_squash**: Public shares 76 | - Public read-only documentation 77 | - Shared resources where individual ownership doesn't matter 78 | - Anonymous FTP-like setups 79 | 80 | You can also combine these with `anonuid` and `anongid` to specify which UID/GID to use for anonymous access: 81 | ```bash 82 | /shared_data 192.168.1.20(rw,all_squash,anonuid=1001,anongid=1001) 83 | ``` -------------------------------------------------------------------------------- /performance-analysis/data/Virtual-vs-Physical-adresses.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/abdoufermat5/unix-and-linux-sysadmin-notes/10b44b43dff4235fd3e4e9755d6f60361093b1e9/performance-analysis/data/Virtual-vs-Physical-adresses.png -------------------------------------------------------------------------------- /performance-analysis/data/lru-workflow.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/abdoufermat5/unix-and-linux-sysadmin-notes/10b44b43dff4235fd3e4e9755d6f60361093b1e9/performance-analysis/data/lru-workflow.png -------------------------------------------------------------------------------- /performance-analysis/data/my-linux-topo.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/abdoufermat5/unix-and-linux-sysadmin-notes/10b44b43dff4235fd3e4e9755d6f60361093b1e9/performance-analysis/data/my-linux-topo.png -------------------------------------------------------------------------------- /performance-analysis/data/performance-analysis-flow.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/abdoufermat5/unix-and-linux-sysadmin-notes/10b44b43dff4235fd3e4e9755d6f60361093b1e9/performance-analysis/data/performance-analysis-flow.png -------------------------------------------------------------------------------- /performance-analysis/training/linux_utemezes.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/abdoufermat5/unix-and-linux-sysadmin-notes/10b44b43dff4235fd3e4e9755d6f60361093b1e9/performance-analysis/training/linux_utemezes.pdf -------------------------------------------------------------------------------- /physical-networking/data/evolution-of-eth.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/abdoufermat5/unix-and-linux-sysadmin-notes/10b44b43dff4235fd3e4e9755d6f60361093b1e9/physical-networking/data/evolution-of-eth.png -------------------------------------------------------------------------------- /physical-networking/readme.md: -------------------------------------------------------------------------------- 1 | # Chapter 14: Physical Networking 2 | 3 | [NOT GONNA SPEND A LOT OF TIME ON THIS] 4 | 5 | ![Wires](https://emproticos.org/wp-content/uploads/2017/04/Presentation1.jpg) 6 | 7 | Physical networking refers to the hardware that connects computers and transmits data. This includes cables, routers, switches, and other devices. While virtual networks are becoming more common, familiarity with traditional networking is still important. 8 | 9 | ## Ethernet 10 | 11 | Having captured over 95% of the world-wide local area network (LAN) market, Ethernet can be found just about everywhere in its many forms. It started as **Bob Metcalfe’s Ph.D. thesis at MIT** but is now described in a variety of IEEE standards. 12 | 13 | Ethernet is the most common physical networking technology. It uses twisted-pair cables to connect devices. Ethernet cables are typically terminated with an RJ-45 connector. Ethernet is used in local area networks (LANs) and wide area networks (WANs). 14 | 15 | ![evolution-of-eth](./data/evolution-of-eth.png) 16 | 17 | ## Wireless 18 | 19 | Wireless networking is becoming more common as technology improves. Wireless networks use radio waves to transmit data. Wireless networks are often used in homes and businesses. They are also used in public places like coffee shops and airports. 20 | 21 | ## SDN: Software Defined Networking 22 | 23 | Software-defined networking (SDN) is a new approach to networking that uses software to control network traffic. SDN allows network administrators to manage network traffic more efficiently. SDN is becoming more common as technology improves. -------------------------------------------------------------------------------- /printing/data/gutenberg printing press.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/abdoufermat5/unix-and-linux-sysadmin-notes/10b44b43dff4235fd3e4e9755d6f60361093b1e9/printing/data/gutenberg printing press.png -------------------------------------------------------------------------------- /printing/readme.md: -------------------------------------------------------------------------------- 1 | # Chapter 12: Printing 2 | 3 | ![gutenberg printing press](gutenberg printing press.png) 4 | 5 | Ages ago, there were three common printing systems: BSD, System V, and CUPS (Common Unix Printing System). Today, Linux and FreeBSD both use CUPS, an up-to-date, sophisticated, network- and security-aware printing system. 6 | 7 | Printing relies on a handful of pieces: 8 | 9 | - A print "spooler" that collects and schedules jobs. The word “spool” originated as an acronym for Simultaneous Peripheral Operation On-Line. 10 | - User-level utilities (cmd or GUI) that talk to the spooler 11 | - Back ends that talk to the printing devices themselves 12 | - A network protocol that lets spoolers communicate and transfer jobs. 13 | 14 | ## CUPS 15 | 16 | CUPS is the most common printing system in use today. It was developed by Michael Sweet at Easy Software Products, which was later acquired by Apple. CUPS is now maintained by Apple. 17 | 18 | CUPS servers are web servers, and the clients are web clients. The CUPS server listens on port 631, and you can access the web interface by pointing your browser to `http://localhost:631`. If you need secure communication with the daemon, you can use `https://localhost:433`. -------------------------------------------------------------------------------- /process-control/data/process-explanation.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/abdoufermat5/unix-and-linux-sysadmin-notes/10b44b43dff4235fd3e4e9755d6f60361093b1e9/process-control/data/process-explanation.png -------------------------------------------------------------------------------- /process-control/data/process-information.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/abdoufermat5/unix-and-linux-sysadmin-notes/10b44b43dff4235fd3e4e9755d6f60361093b1e9/process-control/data/process-information.png -------------------------------------------------------------------------------- /process-control/training/cron.md: -------------------------------------------------------------------------------- 1 | # A cron job to print current date and save it to a file 2 | 3 | Here we've created a cron job that runs every 5 minutes and prints the current date and time to the console. It also saves the date and time to a file called `current-date.txt`. 4 | We've put everything in a bash script called `current-date.sh` and made it executable. 5 | 6 | ```bash 7 | #!/bin/bash 8 | 9 | echo "Current date and time: $(date)" >> current-date.txt 10 | echo "Current date and time: $(date)" 11 | ``` 12 | 13 | ```bash 14 | chmod +x current-date.sh 15 | ``` 16 | 17 | ```bash 18 | crontab -e 19 | ``` 20 | 21 | Then edit the configuration file to create a new cron job that runs every 5 minutes: 22 | 23 | ```bash 24 | */5 * * * * /path/to/current-date.sh 25 | ``` 26 | 27 | ```bash 28 | crontab -l 29 | 30 | */5 * * * * /path/to/current-date.sh 31 | ``` 32 | 33 | Here's what the file `current-date.txt` looks like after a few runs: 34 | 35 | ```bash 36 | cat current-date.txt 37 | Current date and time: 06 mars 2024 17:08:01 CET 38 | Current date and time: 06 mars 2024 17:13:01 CET 39 | Current date and time: 06 mars 2024 17:18:01 CET 40 | Current date and time: 06 mars 2024 17:23:01 CET 41 | ``` 42 | -------------------------------------------------------------------------------- /process-control/training/current-date.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # This script prints the current date and time in a human-readable format 4 | 5 | echo "Current Date and Time: $(date)" >> /tmp/current-date.txt 6 | echo "Current Time: $(date)" -------------------------------------------------------------------------------- /scripts-and-shell/training/awk.md: -------------------------------------------------------------------------------- 1 | # AWK 2 | 3 | The awk command was named using the initials of the three people who wrote the original version in 1977: Alfred Aho, Peter Weinberger, and Brian Kernighan. 4 | 5 | The awk command is a powerful programming language that allows easy manipulation of structured data and the generation of formatted reports. It is a standard feature of most Unix-like operating systems. 6 | 7 | ## Basic Usage 8 | 9 | **awk** is used to filter and manipulate output from other programs and functions. awk works on programs that contain rules comprised of patterns and actions. The action awk takes is executed on the text that matches the pattern. Patterns are enclosed in curly braces ({}). Together, a pattern and an action form a rule. The entire awk program is enclosed in single quotes ('). 10 | 11 | For example, the following command prints the first column of the file `file.txt`: 12 | 13 | ```bash 14 | awk '{print $1}' file.txt 15 | ``` 16 | 17 | The `$1` is a variable that represents the first column of the input. The `print` command prints the value of the variable. 18 | 19 | There a couple of special field identifiers that can be used in awk: 20 | 21 | - `$0` represents the entire line 22 | - `$1` represents the first field 23 | - `$2` represents the second field 24 | - `$3` represents the third field 25 | - `$NF` represents the last field 26 | 27 | We can use the OFS (Output Field Separator) variable to change the output field separator. For example, to print the first and second columns of a file separated by a comma: 28 | 29 | ```bash 30 | awk 'OFS="," {print $1, $2}' file.txt 31 | ``` 32 | 33 | There's also the BEGIN and END patterns that can be used to execute actions before and after processing the input. For example, the following command prints the number of lines in the file `file.txt`: 34 | 35 | ```bash 36 | awk 'END {print NR}' file.txt 37 | ``` 38 | 39 | ## Patterns 40 | 41 | Patterns are used to match lines of input. If a pattern is not specified, the action is executed on every line of input. Patterns can be regular expressions, relational expressions, or any combination of the two. 42 | 43 | For example, the following command prints the lines of the /etc/passwd file where user names start with the letter `a`: 44 | 45 | ```bash 46 | awk -F: '/^a/ {print $1}' /etc/passwd 47 | ``` 48 | 49 | The `-F` option is used to specify the field separator. In this case, the field separator is `:`. 50 | 51 | ## Functions 52 | 53 | awk has a number of built-in functions that can be used to manipulate data. For example, the `length` function returns the length of a string. The following command prints the length of the first field of the file `file.txt`: 54 | 55 | ```bash 56 | awk '{print length($1)}' file.txt 57 | ``` 58 | 59 | The `tolower` and `toupper` functions can be used to convert strings to lowercase and uppercase, respectively. The following command prints the first field of the file `file.txt` in lowercase: 60 | 61 | ```bash 62 | awk '{print tolower($1)}' file.txt 63 | ``` 64 | 65 | The `split` function can be used to split a string into an array. The following command splits the first field of the file `file.txt` into an array and prints the first element of the array: 66 | 67 | ```bash 68 | awk '{split($1, a, "/"); print a[1]}' file.txt 69 | ``` 70 | 71 | ## Write your own scripts 72 | 73 | If your command line gets complicated, or you develop a routine you know you'll want to use again, you can transfer your awk command into a script. 74 | 75 | As example we're going to do all of the following: 76 | 77 | - Tell the shell which executable to use to run the script. 78 | - Prepare awk to use the FS field separator variable to read input text with fields separated by colons (:). 79 | - Use the OFS output field separator to tell awk to use colons (:) to separate fields in the output. 80 | - Set a counter to 0 (zero). 81 | - Set the second field of each line of text to a blank value (it's always an "x," so we don't need to see it). 82 | - Print the line with the modified second field. 83 | - Increment the counter. 84 | - Print the value of the counter. 85 | 86 | ```bash 87 | #!/usr/bin/awk -f 88 | 89 | BEGIN { 90 | # set the input and output field separators 91 | FS=":" 92 | OFS=":" 93 | # zero the accounts counter 94 | accounts=0 95 | } 96 | { 97 | # set field 2 to nothing 98 | $2="" 99 | # print the entire line 100 | print $0 101 | # increment the accounts counter 102 | accounts++ 103 | } 104 | 105 | END { 106 | # print the results 107 | print accounts " accounts.\n" 108 | } 109 | ``` -------------------------------------------------------------------------------- /scripts-and-shell/training/backup_fn.sh: -------------------------------------------------------------------------------- 1 | function backup() { 2 | newname = $1.`date +%Y-%m-%d.%H%M.bak`; 3 | mv $1 $newname; 4 | echo "Backed up $1 to $newname."; 5 | cp -p $newname $1; 6 | } 7 | -------------------------------------------------------------------------------- /scripts-and-shell/training/omit.awk: -------------------------------------------------------------------------------- 1 | #!/usr/bin/awk -f 2 | 3 | BEGIN { 4 | # set the input and output field separators 5 | FS=":" 6 | OFS=":" 7 | # zero the accounts counter 8 | accounts=0 9 | } 10 | { 11 | # set field 2 to nothing 12 | $2="" 13 | # print the entire line 14 | print $0 15 | # increment the accounts counter 16 | accounts++ 17 | } 18 | 19 | END { 20 | # print the results 21 | print accounts " accounts.\n" 22 | } -------------------------------------------------------------------------------- /scripts-and-shell/training/showusage.sh: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | 3 | show_usage() { 4 | echo "Usage: $0 source_dir dest_dir" 1>&2 5 | if [ $# -eq 0 ]; then 6 | exit 99 7 | else 8 | exit $1 9 | fi 10 | } 11 | 12 | if [ $# -ne 2 ]; then 13 | show_usage 14 | else # There are two arguments 15 | if [ -d "$1" ]; then 16 | source_dir="$1" 17 | else 18 | echo "Error: $1 is not a directory." 1>&2 19 | show_usage 20 | fi 21 | if [ -d "$2" ]; then 22 | dest_dir="$2" 23 | else 24 | echo "Error: $2 is not a directory." 1>&2 25 | show_usage 26 | fi 27 | fi 28 | 29 | printf "Copying files from %s to %s\n" "$source_dir" "$dest_dir" 30 | -------------------------------------------------------------------------------- /security/data/client-config.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/abdoufermat5/unix-and-linux-sysadmin-notes/10b44b43dff4235fd3e4e9755d6f60361093b1e9/security/data/client-config.png -------------------------------------------------------------------------------- /security/data/config-files.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/abdoufermat5/unix-and-linux-sysadmin-notes/10b44b43dff4235fd3e4e9755d6f60361093b1e9/security/data/config-files.png -------------------------------------------------------------------------------- /security/data/hash-confirm.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/abdoufermat5/unix-and-linux-sysadmin-notes/10b44b43dff4235fd3e4e9755d6f60361093b1e9/security/data/hash-confirm.png -------------------------------------------------------------------------------- /security/data/port-forwarding.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/abdoufermat5/unix-and-linux-sysadmin-notes/10b44b43dff4235fd3e4e9755d6f60361093b1e9/security/data/port-forwarding.png -------------------------------------------------------------------------------- /security/data/ssh-agent-forwarding.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/abdoufermat5/unix-and-linux-sysadmin-notes/10b44b43dff4235fd3e4e9755d6f60361093b1e9/security/data/ssh-agent-forwarding.png -------------------------------------------------------------------------------- /security/training/OPENVPN in details.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/abdoufermat5/unix-and-linux-sysadmin-notes/10b44b43dff4235fd3e4e9755d6f60361093b1e9/security/training/OPENVPN in details.pdf -------------------------------------------------------------------------------- /security/training/details-view.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/abdoufermat5/unix-and-linux-sysadmin-notes/10b44b43dff4235fd3e4e9755d6f60361093b1e9/security/training/details-view.png -------------------------------------------------------------------------------- /security/training/list-view.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/abdoufermat5/unix-and-linux-sysadmin-notes/10b44b43dff4235fd3e4e9755d6f60361093b1e9/security/training/list-view.png -------------------------------------------------------------------------------- /security/training/params-free.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/abdoufermat5/unix-and-linux-sysadmin-notes/10b44b43dff4235fd3e4e9755d6f60361093b1e9/security/training/params-free.png -------------------------------------------------------------------------------- /security/training/setting-openvpn.md: -------------------------------------------------------------------------------- 1 | # VPN Test Environment with Docker Compose 2 | 3 | This project sets up a VPN test environment using Docker Compose, allowing two remote clients to securely access an internal Ubuntu server and a simple web server on a simulated enterprise LAN. 4 | 5 | ## Project Structure 6 | 7 | ``` 8 | vpn-test/ 9 | ├── docker-compose.yml 10 | ├── openvpn-data/ 11 | │ └── conf/ 12 | ├── webserver/ 13 | │ └── html/ 14 | │ └── index.html 15 | └── ubuntu-server/ 16 | └── client1.ovpn 17 | └── client2.ovpn 18 | ``` 19 | 20 | * **`docker-compose.yml`:** Defines the services (OpenVPN server, web server, Ubuntu server) and their configurations. 21 | * **`openvpn-data/conf/`:** Stores OpenVPN server configuration files, certificates, and keys. 22 | * **`webserver/html/`:** Contains the `index.html` file served by the internal web server. 23 | * **`ubuntu-server/`:** (Optional) You can place scripts or configuration files for the Ubuntu server here. 24 | * **`client1.ovpn`**: OpenVPN configuration file for client 1. 25 | * **`client2.ovpn`**: OpenVPN configuration file for client 2. 26 | 27 | ## Prerequisites 28 | 29 | * Docker Engine and Docker Compose installed on your server machine. 30 | * Basic understanding of networking concepts (IP addresses, subnets, ports). 31 | * At least one another machine for testing. 32 | 33 | ## Setup Steps 34 | 35 | ### 1. Project Directory 36 | 37 | * Create a directory for this project: 38 | 39 | ```bash 40 | mkdir vpn-test 41 | cd vpn-test 42 | ``` 43 | 44 | * Create subdirectories: 45 | 46 | ```bash 47 | mkdir -p openvpn-data/conf webserver/html ubuntu-server 48 | ``` 49 | 50 | ### 2. Docker Compose File 51 | 52 | * Create a file named `docker-compose.yml` with the following content: 53 | 54 | ```yaml 55 | services: 56 | openvpn: 57 | image: kylemanna/openvpn:latest 58 | container_name: openvpn-server 59 | cap_add: 60 | - NET_ADMIN 61 | volumes: 62 | - ./openvpn-data/conf:/etc/openvpn 63 | ports: 64 | - "40000:1194/udp" # Expose a non-standard port (>= 32768 for my Freebox) 65 | networks: 66 | - vpn_network 67 | restart: always 68 | 69 | webserver: 70 | image: nginx:latest 71 | container_name: internal-webserver 72 | volumes: 73 | - ./webserver/html:/usr/share/nginx/html:ro 74 | networks: 75 | - vpn_network 76 | restart: always 77 | 78 | ubuntu-server: 79 | image: ubuntu:latest 80 | container_name: internal-ubuntu 81 | command: tail -f /dev/null # Keep it running 82 | networks: 83 | - vpn_network 84 | restart: always 85 | 86 | networks: 87 | vpn_network: 88 | driver: bridge 89 | ipam: 90 | config: 91 | - subnet: 172.20.0.0/16 92 | ``` 93 | 94 | ### 3. Web Server Content 95 | 96 | * Create a simple `index.html` file in `./webserver/html`: 97 | 98 | ```html 99 | 100 | 101 | 102 | Internal Webserver 103 | 104 | 105 |

Hello from Big local company Webserver!

106 | 107 | 108 | ``` 109 | 110 | ### 4. Generate OpenVPN Configuration 111 | 112 | * **Initialize OpenVPN and Generate Server Config:** 113 | 114 | ```bash 115 | docker-compose run --rm openvpn ovpn_genconfig -u udp://YOUR_PUBLIC_IP:40000 -n 8.8.8.8 -n 8.8.4.4 116 | ``` 117 | 118 | * **Replace `YOUR_PUBLIC_IP` with the actual public IP address of your server.** (use `curl ifconfig.me` to get the public IP or check on google) 119 | * **Adjust the port (40000 here) if you chose a different one.** 120 | * **-n option allows you to set DNS, here we use google's one, but you can use the one you prefer** 121 | 122 | * **Initialize the PKI (Public Key Infrastructure):** 123 | 124 | ```bash 125 | docker-compose run --rm openvpn ovpn_initpki 126 | ``` 127 | 128 | * You'll be prompted to set a passphrase for the CA (Certificate Authority). Choose a strong one and remember it. 129 | 130 | * **Generate Client Certificates:** 131 | 132 | ```bash 133 | docker-compose run --rm openvpn easyrsa build-client-full client1 nopass 134 | docker-compose run --rm openvpn easyrsa build-client-full client2 nopass 135 | ... 136 | ``` 137 | 138 | * **Retrieve Client Configuration Files:** 139 | 140 | ```bash 141 | docker-compose run --rm openvpn ovpn_getclient client1 > client1.ovpn 142 | docker-compose run --rm openvpn ovpn_getclient client2 > client2.ovpn 143 | ... 144 | ``` 145 | 146 | * This will create `client1.ovpn` and `client2.ovpn` in your project directory. 147 | 148 | The `.ovpn` file contains all the necessary parameters to connect to the server such as: 149 | 150 | - Server's address (IP or hostname) and port. 151 | - Protocol (UDP or TCP). 152 | - Certificates and keys. 153 | - Cipher and authentication settings. 154 | - Other options like compression, persistence, etc. 155 | 156 | ### 5. Configure Router (Port Forwarding) 157 | 158 | * If your server is behind a router (like Freebox), you need to configure port forwarding: 159 | * Log in to your router's administration interface (e.g., `192.168.1.254` for Freebox). 160 | * Find the "Port Forwarding" or "Redirection de Ports" settings. 161 | * Create a rule to forward UDP port 40000 (or the port you chose) from the internet to the internal IP address of your server machine on port 40000. 162 | 163 | ![alt text](params-free.png) 164 | ![alt text](list-view.png) 165 | ![alt text](details-view.png) 166 | 167 | ### 6. Configure Firewall (Server) 168 | 169 | * If you have a firewall enabled on your server (e.g., `ufw`), allow incoming traffic on the OpenVPN port: 170 | 171 | ```bash 172 | sudo ufw allow 40000/udp 173 | ``` 174 | 175 | ### 7. Start the Environment 176 | 177 | * Start the Docker Compose services: 178 | 179 | ```bash 180 | docker-compose up -d 181 | ``` 182 | 183 | ### 8. Client Setup (Ubuntu Example for Client 1, my other Ubuntu laptop) 184 | 185 | * **Install OpenVPN:** 186 | 187 | ```bash 188 | sudo apt update 189 | sudo apt install openvpn 190 | ``` 191 | 192 | * **Transfer `client1.ovpn`:** 193 | * Transfer the `client1.ovpn` file to your Ubuntu client machine (using `scp`, a USB drive, or other methods). I've used gmail mhahaha!!! 194 | 195 | * **Connect (Command Line):** 196 | 197 | ```bash 198 | sudo openvpn --config client1.ovpn 199 | ``` 200 | 201 | * Keep the terminal open while connected. 202 | 203 | * **Connect (Network Manager - Optional):** 204 | * Import `client1.ovpn` into Network Manager. 205 | * Connect through the Network Manager interface. 206 | 207 | ### 9. Testing the Connection 208 | 209 | * **Verify IP and Interface:** 210 | * On the client, use `ip addr show` to check for a `tun0` or `tap0` interface with an IP in the 172.20.0.0/16 subnet. 211 | 212 | * **Ping Tests:** 213 | * Ping the web server: `ping 172.20.0.3` 214 | * Ping the Ubuntu server: `ping 172.20.0.4` 215 | 216 | * **Access Web Server:** 217 | * Open a browser on the client and go to `http://172.20.0.3`. 218 | 219 | * **SSH to Ubuntu Server (Optional):** 220 | * `ssh username@172.20.0.4` (replace with the correct username and IP). 221 | 222 | Look at [openvpn in detail.pdf](./OPENVPN%20in%20details.pdf) file to learn more about the core key exchange and tls. 223 | 224 | ## Troubleshooting 225 | 226 | * **TLS Key Negotiation Failed:** 227 | * Verify your server's public IP address in `client1.ovpn`. 228 | * Check firewall rules on the server and router. 229 | * Ensure port forwarding is configured correctly. 230 | * Double-check for typos in commands. 231 | 232 | * **`netcat` Test:** 233 | * On the server, inside the `openvpn` container: `nc -u -l -p 5555` 234 | * On the client: `nc -u YOUR_SERVER_PUBLIC_IP 1194` 235 | * Type messages to see if they are received on the other end. This tests basic UDP connectivity. 236 | 237 | * **Cipher Issue (Warning):** 238 | * Add `data-ciphers-fallback BF-CBC` to both `server.conf` and `client1.ovpn` if needed (less likely to be the cause of major problems). 239 | 240 | * **Other Issues:** 241 | * Examine OpenVPN server logs: `docker-compose logs openvpn` 242 | * Check client-side logs. 243 | * Consult OpenVPN documentation and forums. 244 | 245 | ## Security Considerations (for Production) 246 | 247 | * **Stronger Authentication:** Use certificate-based authentication with strong ciphers. 248 | * **Firewall:** Implement a robust firewall on your server. 249 | * **Intrusion Detection:** Set up an IDS to monitor for suspicious activity. 250 | * **Regular Updates:** Keep all software up to date. 251 | * **Principle of Least Privilege:** Grant users only the necessary access. 252 | 253 | ## Disclaimer 254 | 255 | This guide is for educational and testing purposes. It provides a basic setup and may not cover all security aspects required for a production environment. Adapt and enhance the configuration based on your specific security needs and best practices. 256 | -------------------------------------------------------------------------------- /security/training/vpn-test/docker-compose.yml: -------------------------------------------------------------------------------- 1 | services: 2 | openvpn: 3 | image: kylemanna/openvpn:latest 4 | container_name: openvpn-server 5 | cap_add: 6 | - NET_ADMIN 7 | volumes: 8 | - ./openvpn-data/conf:/etc/openvpn 9 | ports: 10 | - "40000:1194/udp" 11 | networks: 12 | - vpn_network 13 | restart: always 14 | 15 | webserver: 16 | image: nginx:latest 17 | container_name: internal-webserver 18 | volumes: 19 | - ./webserver/html:/usr/share/nginx/html:ro 20 | networks: 21 | - vpn_network 22 | restart: always 23 | 24 | ubuntu-server: 25 | image: ubuntu:latest 26 | container_name: internal-ubuntu 27 | command: tail -f /dev/null # Keep it running 28 | networks: 29 | - vpn_network 30 | restart: always 31 | 32 | networks: 33 | vpn_network: 34 | driver: bridge 35 | ipam: 36 | config: 37 | - subnet: 172.20.0.0/16 -------------------------------------------------------------------------------- /security/training/vpn-test/webserver/html/index.html: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | Internal Webserver 5 | 6 | 7 |

Hello from the Internal Webserver!

8 | 9 | -------------------------------------------------------------------------------- /single-sign-on/data/SSO-components.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/abdoufermat5/unix-and-linux-sysadmin-notes/10b44b43dff4235fd3e4e9755d6f60361093b1e9/single-sign-on/data/SSO-components.png -------------------------------------------------------------------------------- /single-sign-on/data/common-attrs.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/abdoufermat5/unix-and-linux-sysadmin-notes/10b44b43dff4235fd3e4e9755d6f60361093b1e9/single-sign-on/data/common-attrs.png -------------------------------------------------------------------------------- /single-sign-on/data/pam-example.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/abdoufermat5/unix-and-linux-sysadmin-notes/10b44b43dff4235fd3e4e9755d6f60361093b1e9/single-sign-on/data/pam-example.png -------------------------------------------------------------------------------- /single-sign-on/data/sssd-conf-file.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/abdoufermat5/unix-and-linux-sysadmin-notes/10b44b43dff4235fd3e4e9755d6f60361093b1e9/single-sign-on/data/sssd-conf-file.png -------------------------------------------------------------------------------- /single-sign-on/readme.md: -------------------------------------------------------------------------------- 1 | # Chapter 17: Single Sign-On 2 | 3 | ![Single Sign-On](https://pbs.twimg.com/media/F5RqTpkXYAAZnUB.jpg) 4 | 5 | Both users and system administrators would like account information to magically propagate to all an environment’s computers so that a user can log in to any system with the same credentials. The common term for this feature is “single sign-on” (SSO), and the need for it is universal. 6 | 7 | SSO involves two core security concepts: identity and authentication. A user identity is the abstract representation of an individual who needs access to a system or an application. It typically includes attributes such as a username, password, user ID, and email address. Authentication is the act of proving that an individual is the legitimate owner of an identity. 8 | 9 | ## Core SSO elements 10 | 11 | Four core elements are essential for SSO: 12 | 13 | 1. A central directory store that contains user identity and authorization information. The most common solutions are directory services based on the Lightweight Directory Access Protocol (LDAP). In environments that mix Windows and Unix systems, the central directory store is often Microsoft Active Directory which includes a customized, nonstandard version of LDAP. 14 | 2. A tool for managing user information in the directory. For native LDAP implementations, phpLDAPadmin or Apache Directory Studio are popular choices. For Active Directory, the Microsoft Management Console (MMC) is the standard tool. 15 | 3. A mechanism for authenticating user identities. You can authenticate users directly against an LDAP store, but it's also common to use the Kerberos ticket-based authentication system. In windows AD uses a customized version of Kerberos. 16 | Authentication on modern UNIX and Linux systems goes through the Pluggable Authentication Module system, aka PAM. You can use the System Security Services Daemon (sssd) to aggregate access to user identity and authentication services, then point PAM at sssd. [Check this link for more info about PAM](https://www.redhat.com/sysadmin/pluggable-authentication-modules-pam) 17 | 4. Centralized-identity-and-authentication-aware versions of the C library routines that look up attributes. 18 | 19 | THe following example uses Active Directory as the directory server. Note that both time synchronization (NTP) and hostname mapping (DNS) are critical for env that uses kerberos because authentication tickets are time stamped and have a limited validity period. 20 | 21 | ![SSO-components](./data/SSO-components.png) 22 | 23 | ## LDAP 24 | 25 | A directory service is just a database, but one that makes a few assumptions. Any kind of data that matches the assumptions is a candidate for inclusion in the directory. 26 | 27 | The basic assumptions are as follows: 28 | 29 | - Data onjects are relatively small. 30 | - The database will be widely replicated and cached. 31 | - The information is attribute-based. 32 | - Data are read often but written infrequently. 33 | - Searching is a common operation. 34 | 35 | Ironically, LDAP is anything but lightweight. It was originally a gateway protocol that allowed TCP/IP clients to talk to an older directory service called X.500 (obsolete now). 36 | 37 | Microsoft’s Active Directory is the most common instantiation of LDAP, and many sites use it for both Windows and Unix systems. For environments that are Unix-only, OpenLDAP is a popular choice. 38 | 39 | ### Uses for LDAP 40 | 41 | The most common use of LDAP is to act as a central repository for login names, passwords, and account attributes. However, LDAP can be used in many other ways: 42 | 43 | - It can store additional information about users, such as email addresses, phone numbers, and addresses. 44 | - Most mail systems --- including **sendmail**, **Exim**, **Postfix** --- can draw a large part of their routing information from LDAP. 45 | - LDAP makes it easy for apps to authenticate users without having to store passwords in the app’s database. 46 | - LDAP is well supported by common scripting languages such as Perl and Python 47 | 48 | ### Structure of LDAP data 49 | 50 | LDAP data are property lists known as "entries". Each entry consists of a set of named attributes along with those attribute's values. 51 | 52 | As an example, here’s a typical (but simplified) /etc/passwd line expressed as an LDAP entry: 53 | 54 | ```text 55 | dn: uid=jdoe,ou=users,dc=abacus,dc=net 56 | objectClass: top 57 | objectClass: person 58 | objectClass: organizationalPerson 59 | objectClass: inetOrgPerson 60 | objectClass: posixAccount 61 | objectClass: shadowAccount 62 | uid: jdoe 63 | cn: John Doe 64 | userPassword: {crypt}$1$dflnkzndcsbez&33565Vdvd5p0 65 | loginShell: /bin/bash 66 | uidNumber: 1001 67 | gidNumber: 1001 68 | homeDirectory: /home/jdoe 69 | ``` 70 | 71 | This notation is a simple example of LDIF (LDAP Data Interchange Format). Entries are organized into a hierarchy through the use of "distinguished names" that form a sort of search path. As in DNS, the "most significant bit" goes on the right. In the example above, the DNS name `abacus.net` has structured the top levels of the LDAP hierarchy. It has been broken down into 2 domain components (`dc`'s), `abacus` and `net`. 72 | 73 | LDAP entries are typically schematized through the use of an objectClass attribute. Object classes specify the attributes that an entry can contain, some of which may be required for validity. The schemata also assign a data type to each attribute. Object classes nest and combine in the traditional OO fashion. 74 | 75 | ![common-attrs](./data/common-attrs.png) 76 | 77 | ### OpenLDAP 78 | 79 | In the OpenLDAP distribution, slapd is the standard LDAP server daemon. In an environment with multiple servers, `slurpd` runs on the master server and replicates changes to the slave servers. 80 | 81 | The setup is straightforward: 82 | 83 | Create an `/etc/openldap/slapd.conf` file that contains the server’s configuration. 84 | 85 | ```bash 86 | database bdb 87 | suffix "dc=abacus,dc=net" 88 | rootdn "cn=admin,dc=abacus,dc=net" 89 | rootpw {crypt}xjsifuFDGRs 90 | directory /var/lib/ldap 91 | ``` 92 | 93 | The database format defaults to Berkeley DB. The suffix is the top of the LDAP hierarchy similar to DNS root domain. The rootdn is the distinguished name of the root user. The rootpw is the root user’s password. The directory is where the database files are stored. 94 | 95 | ## Using directory services for login 96 | 97 | Once you have a directory service set up, complete the following configuration chores so your system can enter SSO paradise: 98 | 99 | - If you are planning to use AD with Kerberos, configure Kerberos and join the system to the AD domain. 100 | - Configure sssd to communicate with the appropriate identity and authentication services(AD, LDAP, or Kerberos). 101 | - Configure the name service switch, `/etc/nsswitch.conf`, to use sssd for user and group information. 102 | - Configure PAM to use sssd for authentication. 103 | 104 | SOme use the traditional `getpwent` family of library routines to look up user information, whereas others use the `nsswitch` mechanism to determine which library to use. The `nsswitch` mechanism is a simple configuration file, `/etc/nsswitch.conf`, that tells the system which library to use for each type of information. 105 | 106 | ### Kerberos 107 | 108 | Kerberos is a ticket-based authentication system that uses symmetric key cryptography. The debut of `realmd` has made the task of joining a Linux system to an Active Directory domain much easier. `realmd` act as a configuration tool for sssd and Kerberos. 109 | 110 | Before joining an AD domain, make sure the following are in place: 111 | 112 | - `realmd` is installed on the Linux system. 113 | - `sssd` is installed. 114 | - `ntpd` is installed and running. 115 | - You know the correct name of the AD domain. 116 | - You have the credentials of a user who has permission to join the domain. 117 | 118 | For example, to join the `abacus.net` domain, and the authorized user is `admin_user`, run the following command: 119 | 120 | ```bash 121 | sudo realm join abacus.net -U admin_user 122 | 123 | # then verify with 124 | realm list 125 | ``` 126 | 127 | ### SSSD: System Security Services Daemon 128 | 129 | The UNIX and Linux road to SSO nirvana has been a rough one. Years ago, it was common to set up independent authentification system for every service or app. This 130 | approach often resulted in a morass of separate configurations and undocumented dependencies that were impossible to manage over time. Users’ passwords would work with one application but not another, causing frustration for everyone. 131 | 132 | Microsoft formerly published extensions (originally called “Services for UNIX,” then “Windows Security and Directory Services for UNIX,” and finally, “Identity Management for UNIX” in Windows Server 2012) that facilitated the housing of UNIX users and groups within Active Directory. Putting the authority for managing these attributes in a non-UNIX system was an unnatural fit, however. To the relief of many, Microsoft discontinued this feature as of Windows Server 2016. 133 | 134 | These issues needed some kind of comprehensive solution, and that’s just what we got with `sssd`. sssd is a one-stop shop for user identity wrangling, authentication, and account mapping. It can also cache credentials off-line, which is useful for mobile devices. sssd supports authentication both through native LDAP and through Kerberos. 135 | 136 | Here's an example of `sssd` configuration file: 137 | 138 | ```bash 139 | [sssd] 140 | services = nss, pam 141 | domains = LDAP 142 | 143 | [domain/LDAP] 144 | id_provider = ldap 145 | auth_provider = ldap 146 | ldap_uri = ldap://ldap.abacus.net 147 | ldap_user_search_base = dc=abacus,dc=net 148 | tls_reqcert = demand 149 | ldap_tls_cacert = /etc/ssl/certs/ca-certificates.crt 150 | ``` 151 | 152 | For obvious security reasons, sssd does not allow authentication over an unencrypted channel, so the use of LDAPS/TLS is required. Setting the tls_reqcert attribute to demand in the example above forces sssd to validate the server certificate as an additional check. sssd drops the connection if the certificate is found to be deficient. 153 | 154 | There's three concepts in `sssd`: 155 | 156 | - **The Monitor**: This is the main daemon that controls the other daemons. It reads the configuration file and starts the other daemons. 157 | - **The Providers**: These are modules with specific auth backend awareness. They are responsible for the actual authentication and identity lookups. 158 | - **Responders**: They interact with Linux and implement features. 159 | 160 | ![sssd-conf-file](./data/sssd-conf-file.png) 161 | 162 | ### PAM 163 | 164 | The PAM system relieves programmers of the chore of implementing direct connections to authentication systems and gives sysadmins flexible, modular control over the system's authentication methods. 165 | 166 | In the distant past, commands like `login` included hardwired authentication code that prompted the user for a password and checked it against the encrypted password in `/etc/shadow` (or `/etc/passwd` in the old days). It was impossible to change the authentication method without recompiling the program and administrators had little or no control over details such as whether the system should accept “password” as a valid password.. PAM was created to solve this problem. 167 | 168 | PAM puts the system’s authentication routines into a shared library that login and other programs can call. By separating authentication functions into a discrete subsystem, PAM makes it easy to integrate new advances in authentication and encryption. For instance, MFA (multi-factor authentication) can be added to a system without changing the login program. 169 | 170 | PAM is configured through a series of files in the `/etc/pam.d` directory. Each file corresponds to a specific service or application. The files contain a series of lines that define the authentication steps that PAM should take when the service is invoked. 171 | 172 | The general format of a line in a PAM configuration file is: 173 | 174 | ```text 175 | module-type control-flag module-path [ arguments ] 176 | ``` 177 | 178 | The order of the modules in the file is important (prompting for a password before checking it, for example). 179 | 180 | The `module-type` is the type of module being called. The most common types are: `auth`, `account`, `password`, and `session`. `auth` modules identify the user and grant group membership. Modules that do `account` chores enforce restrictions such as limiting logins to particular time of day, limiting the number of simultaneous users, or limiting the ports on which logins can occur. (For example, you would use an `account`-type module to restrict root logins to the console.) `session` chores includes tasks that are done before, or after a user is granted access; for example mouting a user's home directory. Finally `password` modules change a user's passwor or passphrase. 181 | 182 | The `control-flag` specifies how the modules in the stack should interact to produce an ultimate result for the stack. The most common control flags are `include`, `required`, `requisite`, `sufficient`, and `optional`. The `include` flag is used to include the configuration of another service. The `required` flag means that the module must succeed for the stack to succeed. The `requisite` flag means that the module must succeed, but if it fails, the stack fails immediately. The `sufficient` flag means that if the module succeeds, the stack succeeds immediately (HAHAHA! LIES). The `optional` flag means that the module is not required for the stack to succeed. 183 | 184 | Example of `/etc/pam.d/login`: 185 | 186 | ![pam-example](./data/pam-example.png) 187 | 188 | The auth stack includes several modules. On the first line, the pam_nologin module checks for the existence of a `/etc/nologin` file. If the file exists, the module aborts the login immediately unless the user is root. The pam_securetty module ensures that root can only log on terminals listed in `/etc/securetty`. `pam_env` sets up the user’s environment variables. The pam_unix module checks the user’s password against the encrypted password in `/etc/shadow`. If the user doesn’t have a local UNIX account, `pam_sss` attempts to authenticate the user against the sssd service. 189 | 190 | ## LDAP alternatives 191 | 192 | - **NIS**: The Network Information Service (NIS) is a simple directory service that predates LDAP. It is still in use in some environments (FreeBSD, for example). 193 | - **rsync**: The `rsync` command can be used to synchronize files between systems. It is not a directory service, but it can be used to keep files in sync. Here's an example of how to use `rsync` to keep the `/etc/passwd` and `/etc/shadow` files in sync between two systems: 194 | 195 | ```bash 196 | rsync -gopt -e ssh /etc/passwd /etc/shadow abdoufermat:/etc 197 | ``` 198 | 199 | The -gopt options preserve the permissions, ownerships, and modification times of the file. 200 | -------------------------------------------------------------------------------- /single-sign-on/training/genuine-dialog.md: -------------------------------------------------------------------------------- 1 | # Breakdown the shit Bob! 2 | 3 | Here's a dialogue between Bob, an IT expert, and Jane, an intern, designed to explain key identity and access management concepts in a relatable way: 4 | 5 | **Scene:** Bob's office. A whiteboard is covered in diagrams and acronyms. Jane looks slightly overwhelmed. 6 | 7 | **Jane:** Bob, I've been reading about user authentication and it's... a lot. I keep seeing all these acronyms – SSO, AD, LDAP... it's like alphabet soup! 8 | 9 | **Bob:** (Chuckles) It can definitely feel that way at first. But these are all tools that make life easier, both for users and for us IT folks. Want me to break it down a bit? 10 | 11 | **Jane:** Please! I'm especially confused about how they all fit together. 12 | 13 | **Bob:** Okay, let's start with the big picture. Imagine you work for a big company, right? You have a ton of different applications and systems you need to access – email, your work files, maybe some specialized software. Logging into each one separately would be a pain, wouldn't it? 14 | 15 | **Jane:** Totally. 16 | 17 | **Bob:** That's where Single Sign-On (SSO) comes in. It's like having a master key that unlocks all the doors you need. You log in once, and boom – you're authenticated for all your applications. 18 | 19 | **Jane:** That sounds great! But how does it actually work? 20 | 21 | **Bob:** There are a few ways, but one common one is using a directory service. Think of it like a giant phonebook for your organization. It stores information about all the users, groups, and even computers. Two common directory services are Active Directory (AD) and LDAP. 22 | 23 | **Jane:** So, AD and LDAP are just different types of phonebooks? 24 | 25 | **Bob:** Exactly! AD is Microsoft's version, very popular in corporate environments. LDAP is an open standard, more flexible but maybe a bit more complex. Both store data in a hierarchical way – you have your organization at the top, then departments, then individual users. 26 | 27 | **Jane:** Okay, I think I'm getting that. But what about PAM, NSS, and SSSD? I saw those in my reading too. 28 | 29 | **Bob:** PAM (Pluggable Authentication Modules) is kind of like a toolkit for authentication methods. It lets you choose how you want users to prove they are who they say they are – passwords, smart cards, fingerprints, whatever. 30 | 31 | **Jane:** Like different locks for the doors? 32 | 33 | **Bob:** (Nods) NSS (Name Service Switch) is a way for Linux systems to look up user information. It can use different sources – local files, LDAP, even NIS (the older cousin of LDAP). 34 | 35 | **Jane:** And SSSD? 36 | 37 | **Bob:** SSSD (System Security Services Daemon) is kind of like a bridge between your Linux system and your directory service. It caches user information locally, so access is faster, and it handles the authentication process. 38 | 39 | **Jane:** This is starting to make more sense. But there's still one more – Kerberos. That one sounds like a mythical creature. 40 | 41 | **Bob:** (Laughs) It's actually named after a three-headed dog from Greek mythology! Kerberos is an authentication protocol that uses tickets to prove your identity. It's super secure and works well in large, complex networks. 42 | 43 | **Jane:** Wow, this is a lot to take in, but I think I have a much better grasp now. Thanks for breaking it down, Bob! 44 | 45 | **Bob:** No problem, Jane! Feel free to ask if anything else comes up. This stuff can be tricky, but it's all about keeping our systems and data safe, so it's important to understand. 46 | 47 | Made by Google's Gemini Advanced model -------------------------------------------------------------------------------- /smb/data/carnaval-mindelo-defile-ecole-samba-tropicale-1620x600.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/abdoufermat5/unix-and-linux-sysadmin-notes/10b44b43dff4235fd3e4e9755d6f60361093b1e9/smb/data/carnaval-mindelo-defile-ecole-samba-tropicale-1620x600.jpg -------------------------------------------------------------------------------- /smb/data/danse-samba.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/abdoufermat5/unix-and-linux-sysadmin-notes/10b44b43dff4235fd3e4e9755d6f60361093b1e9/smb/data/danse-samba.jpg -------------------------------------------------------------------------------- /smb/data/smb-history.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/abdoufermat5/unix-and-linux-sysadmin-notes/10b44b43dff4235fd3e4e9755d6f60361093b1e9/smb/data/smb-history.png -------------------------------------------------------------------------------- /smb/data/smb-vs-nfs.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/abdoufermat5/unix-and-linux-sysadmin-notes/10b44b43dff4235fd3e4e9755d6f60361093b1e9/smb/data/smb-vs-nfs.png -------------------------------------------------------------------------------- /smb/readme.md: -------------------------------------------------------------------------------- 1 | # Chapter 22: SMB - Server Message Block 2 | 3 | ![intro-samba](./data/danse-samba.jpg) 4 | 5 | NFS covers the most popular system for sharing files among UNIX and Linux systems. However, UNIX systems also need to share files with Windows systems. This is where SMB comes in. 6 | 7 | ![smb-history](./data/smb-history.png) 8 | Although it’s common within the industry to refer to SMB fileshares as CIFS, the truth is that CIFS was deprecated long ago; only SMB lives on. 9 | 10 | ## SAMBA: SMB server for Linux 11 | 12 | Samba is the server side implementation of SMB on UNIX and Linux systems. The real beauty of Samba is that you install only one package on the server side; no special software is required on the Windows side. 13 | 14 | In the Windows world, a filesystem or directory made available over the network is known as a "share" (niggeuhh wheeeutt??) 15 | 16 | Samba can also implement a variety of other cross-platform services other than file sharing: 17 | 18 | - Authentication & authorization 19 | - Network printing 20 | - Name resolution 21 | - Service announcement (file server and printer discovery) 22 | 23 | Most of Samba’s functionality is implemented by two daemons, `smbd` and `nmbd`. `smbd` implements file and print services as well as authentication and authorization. `nmbd` is responsible for the other major SMB components: name resolution and service announcement. 24 | 25 | Unlike NFS, which requires kernel-level support, Samba requires no drivers or kernel modifications and runs entirely as a user process. It binds to the sockets used for SMB requests and waits for a client request access to a resource.Once a request has been authenticated, `smbd` forks an instance of itself that runs as the user who is making the requests. As a result, all normal file access permissions (including group permissions) are obeyed. The only special functionality that `smbd` adds on top of this is a file locking service that gives Windows systems the locking semantics to which they are accustomed. 26 | 27 | ALmost all OSs support SMB. 28 | 29 | ![smb-vs-nfs](./data/smb-vs-nfs.png) 30 | 31 | ## Installing and configuring Samba 32 | 33 | Most Linux distributions include Samba by default. 34 | 35 | You configure Samba by editing the `/etc/samba/smb.conf` file (`/usr/local/etc/smb4.conf` on FreeBSD). The file specifies the directories to share, their access rights, and Samba’s general operational parameters (type `testparm -v` to see all config options) 36 | 37 | The most common use of Samba is to share files with Windows clients. Access to these shares must be authenticated through a user account by one of two options. 38 | 39 | ### File-sharing with local authentication 40 | 41 | The simplest way to authenticate users who want to access Samba shares is by creating a local account for them on the UNIX or Linux server. 42 | 43 | As example in `smb.conf` file: 44 | 45 | ```bash 46 | [global] 47 | workgroup = ulsah 48 | security = user # this tells Samba to use local accounts for authentication 49 | netbios name = freebsd-book 50 | ``` 51 | 52 | Samba has its own command, `smbpasswd`, for setting up Windows-style password hashes. 53 | 54 | ```bash 55 | $ sudo smbpasswd -a abdou 56 | New SMB password: 57 | Retype new SMB password: 58 | ``` 59 | 60 | ### File-sharing with Active Directory authentication 61 | 62 | It's better to have a centralized authentication system like Active Directory (AD) to manage user accounts. With `sssd` it's now easier to integrate Samba with AD on UNIX/Linux. 63 | 64 | ```bash 65 | [global] 66 | workgroup = ulsah 67 | realm = ulsah.example.com 68 | security = ads 69 | dedicated keytab file = FILE:/samba/samba.keytab 70 | kerberos method = dedicated keytab 71 | ``` 72 | 73 | The dedicated keytab file and kerberos method parameters enable Samba to work properly with Active Directory’s Kerberos implementation. Each share that you expose needs its own stanza in the configuration file (`abdoushare` in the example above) 74 | 75 | ### Configuring shares 76 | 77 | We can configure through `/etc/samba/smb.conf` which directories should be shared. 78 | 79 | Example: 80 | 81 | ```bash 82 | [abdoushare] 83 | path = /home/asadiakhou/youtube_downloads 84 | browseable = yes 85 | read only = no 86 | create mask = 0777 # all rights 87 | directory mask = 0777 # everything 88 | public = yes # everyone 89 | guest ok = yes 90 | ``` 91 | 92 | Here, SMB clients see a mountable share named `\\sambaserver\abdoushare`. 93 | 94 | We can use the `homes` stanza to convert users home directories into distinct smb shares. 95 | 96 | ```bash 97 | [homes] 98 | comment = Home Directories 99 | browsable = no 100 | valid_user = %S 101 | read_only = no 102 | ``` 103 | 104 | The %S expand to the username associated with each share, restricting access to the owner of the home directory. 105 | 106 | Samba uses its magic [homes] section as a last resort. If a particular user’s home directory has ancexplicitly defined share in the configuration file, the parameters set there override the values set through [homes]. 107 | 108 | We can share projects with samba, allowing only members of the group to mount the share. 109 | 110 | ```text 111 | [eng] 112 | comment = Group SHare for engineering 113 | valid_user = @eng 114 | path = /home/eng 115 | ; Disable ACLs (too complicated to handle) 116 | nt acl support = no 117 | 118 | : sensible permissions on all files and setgid bit set for dirs 119 | create mask = 0660 120 | directory mask = 2770 121 | force directory mode = 2000 122 | force group = eng 123 | 124 | browseable = no 125 | read_only = no 126 | guest ok = no 127 | ``` 128 | 129 | With this if user Ben in the `eng` team can mount the smb share on his pc. 130 | If Ben create a new file in the share `home/eng` this will have `-rw-rw---- 1 ben eng 1024 Nov 24 14:30 remote_file.txt` permissions. 131 | If Ben create a dir called `projectBen` Samba will apply the `directory mask = 2770` and `force directory mode = 2000` rule: 132 | 133 | ``` 134 | drwxrws--- 2 ben eng 4096 Nov 24 14:35 projectBen 135 | ``` 136 | 137 | The setgid (`s`) ensures all files or directories created inside `projectBen` automatically belong to the `eng` group. 138 | 139 | If Allan join the projectBen and create a file `test.txt` this would be listed as `-rw-rw---- 1 allan eng 512 Nov 24 15:45 test.txt`. 140 | 141 | ## Mounting SMB file shares 142 | 143 | Mounting for SMB file shares works quite differently from how it’s done for other network filesystems. In particular, SMB volumes are mounted by a specific user rather than being mounted by the system itself. 144 | 145 | On linux: `sudo mount -t cifs -o username=abdou //smb-server/eng /home/abdou/eng` 146 | 147 | In UNIX/Linux, a mounted network share is viewed as a system-wide resource, typically owned by the user who mounts it (often root unless you specify uid,gid,fmask or dmask options), without associating it with individual users. In contrast, Windows treats network shares as user-specific, with access controlled by each user's permissions. 148 | 149 | ## Browsing SMB file shares 150 | 151 | Samba includes a command-line utility called `smbclient` that lets you list file shares without actually mounting them. It also define an FTP-like interface for interactive access. 152 | 153 | Once you're connected type `help`. 154 | 155 | ![chill guy](https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcS_9bE7txw_Nqjl-JAli54YuH1zcJMEsuMFIg&s) 156 | 157 | ## Ensuring Samba security 158 | 159 | It’s important to be aware of the security implications of sharing files and other resources over a network. For a typical site, you need to do two things to ensure a basic level of security: 160 | 161 | - Explicitly specify which clients can access the resources shared by Samba (`hosts allow` and `hosts deny`). 162 | 163 | - Block access to the server from outside your organization. Samba does not use encryption for its data transport (only for password authentication). 164 | 165 | Blocking is typically implemented at the network firewall level. 166 | 167 | ## Debugging Samba 168 | 169 | To debug Samba you can consult two sources: `smbstatus` command or Samba's logging facilities. 170 | 171 | If something wrong check the `smb.conf` options and adjust! 172 | -------------------------------------------------------------------------------- /smb/training/sambaxp-2015-cloudy-future.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/abdoufermat5/unix-and-linux-sysadmin-notes/10b44b43dff4235fd3e4e9755d6f60361093b1e9/smb/training/sambaxp-2015-cloudy-future.pdf -------------------------------------------------------------------------------- /smb/training/smb-hands-on/Dockerfile: -------------------------------------------------------------------------------- 1 | # Base image 2 | FROM debian:latest 3 | 4 | # copy conf 5 | COPY eng.conf /etc/samba/eng.conf 6 | 7 | # Install Samba and smbclient 8 | RUN apt-get update && apt-get install -y samba smbclient 9 | 10 | # Create shared directory, users, and groups 11 | RUN mkdir -p /share/eng && \ 12 | groupadd eng && \ 13 | useradd -m ben -G eng && \ 14 | useradd -m allan -G eng && \ 15 | chmod -R 2770 /share/eng && \ 16 | chown -R root:eng /share/eng 17 | 18 | # Set Samba passwords for users 19 | # RUN echo -e "benpwd123\nbenpwd123" | smbpasswd -s -a ben && \ 20 | # echo -e "allanpwd123\nallanpwd123" | smbpasswd -s -a allan 21 | 22 | # Append the shared configuration to smb.conf 23 | RUN cat /etc/samba/eng.conf >> /etc/samba/smb.conf 24 | 25 | # Expose Samba ports 26 | EXPOSE 139 445 27 | 28 | # Start Samba service 29 | CMD ["sh", "-c", "service smbd start && tail -f /dev/null"] 30 | -------------------------------------------------------------------------------- /smb/training/smb-hands-on/Dockerfile.clients: -------------------------------------------------------------------------------- 1 | # Use Debian as the base image 2 | FROM debian:latest 3 | 4 | # Install necessary tools (e.g., smbclient) 5 | RUN apt-get update && apt-get install -y smbclient iproute2 6 | 7 | # Set the working directory 8 | WORKDIR /root 9 | 10 | # This command will run when the container starts and keep it running interactively 11 | CMD ["tail", "-f", "/dev/null"] 12 | -------------------------------------------------------------------------------- /smb/training/smb-hands-on/docker-compose.yml: -------------------------------------------------------------------------------- 1 | version: '3.9' 2 | 3 | services: 4 | smb-server: 5 | build: 6 | context: . 7 | dockerfile: Dockerfile 8 | container_name: smb-server 9 | networks: 10 | trusted_network: 11 | 12 | ben-pc: 13 | build: 14 | context: . 15 | dockerfile: Dockerfile.clients 16 | container_name: ben-pc 17 | networks: 18 | trusted_network: 19 | 20 | 21 | allan-pc: 22 | build: 23 | context: . 24 | dockerfile: Dockerfile.clients 25 | container_name: allan-pc 26 | networks: 27 | trusted_network: 28 | 29 | 30 | malicious-user-pc: 31 | build: 32 | context: . 33 | dockerfile: Dockerfile.clients 34 | container_name: malicious-user-pc 35 | networks: 36 | untrusted_network: 37 | 38 | 39 | networks: 40 | trusted_network: 41 | driver: bridge 42 | untrusted_network: 43 | driver: bridge 44 | -------------------------------------------------------------------------------- /smb/training/smb-hands-on/eng.conf: -------------------------------------------------------------------------------- 1 | [eng] 2 | comment = Group Share for engineering 3 | path = /share/eng 4 | valid users = @eng 5 | nt acl support = no 6 | create mask = 0660 7 | directory mask = 2770 8 | force directory mode = 2000 9 | force group = eng 10 | browseable = no 11 | read only = no 12 | guest ok = no -------------------------------------------------------------------------------- /smb/training/smb-hands-on/homes.conf: -------------------------------------------------------------------------------- 1 | [homes] 2 | comment = Home Directories 3 | browsable = no 4 | valid_user = %S 5 | read_only = no -------------------------------------------------------------------------------- /smb/training/smb-hands-on/readme.md: -------------------------------------------------------------------------------- 1 | 2 | # **Samba Hands on Lab** 3 | 4 | This lab simulates a basic network with a Samba server and multiple client machines. The setup involves two network segments: 5 | 6 | - **Trusted Network**: Ben’s and Allan’s PCs 7 | - **Untrusted Network**: Malicious User's PC 8 | 9 | The goal of this setup is to test different network interactions, including Samba file sharing and security configurations between trusted and untrusted machines. 10 | 11 | ### **Lab Setup Overview** 12 | 13 | - **Samba Server (`smb-server`)**: 14 | - Provides a file share that is accessible by trusted clients (Ben and Allan). 15 | - Configured with the `eng` share that is only accessible by members of the `eng` group. 16 | 17 | - **Ben’s PC (`ben-pc`)**: 18 | - A trusted machine that has access to the `eng` share from the Samba server. 19 | 20 | - **Allan’s PC (`allan-pc`)**: 21 | - Another trusted machine that has access to the `eng` share. 22 | 23 | - **Malicious User’s PC (`malicious-user-pc`)**: 24 | - A machine on an untrusted network, isolated from the Samba server and the trusted machines. 25 | 26 | ### **Requirements** 27 | - Docker 28 | - Docker Compose 29 | 30 | ### **Setup Instructions** 31 | 32 | 1. **Clone the Repository** (if applicable): 33 | ```bash 34 | git clone 35 | cd 36 | ``` 37 | 38 | 2. **Build and Start the Containers**: 39 | In the project directory (where `docker-compose.yml` is located), run the following command to build and start the containers: 40 | ```bash 41 | docker-compose up --build -d 42 | ``` 43 | 44 | 3. **Accessing the Containers**: 45 | Once the containers are up and running, you can interact with each of the services: 46 | 47 | - **Samba Server**: You can check the server’s status and configuration inside the `smb-server` container. 48 | ```bash 49 | docker exec -it smb-server bash 50 | ``` 51 | 52 | - **Ben’s PC**: To interact with Ben’s PC container, use: 53 | ```bash 54 | docker exec -it ben-pc bash 55 | ``` 56 | 57 | - **Allan’s PC**: Similarly, access Allan’s PC: 58 | ```bash 59 | docker exec -it allan-pc bash 60 | ``` 61 | 62 | - **Malicious User’s PC**: Access the malicious user’s machine to simulate untrusted behavior: 63 | ```bash 64 | docker exec -it malicious-user-pc bash 65 | ``` 66 | 67 | 4. **Interacting with the Samba Server**: 68 | - From **Ben’s PC** and **Allan’s PC**, you should be able to access the shared folder from the Samba server: 69 | ```bash 70 | smbclient //smb-server/eng -U ben 71 | ``` 72 | or 73 | ```bash 74 | smbclient //smb-server/eng -U allan 75 | ``` 76 | 77 | - **Malicious User’s PC** should not have access to the Samba share if configured correctly. 78 | 79 | 5. **Stopping the Containers**: 80 | To stop the containers, run the following command: 81 | ```bash 82 | docker-compose down 83 | ``` 84 | 85 | --- 86 | 87 | ### **Network Configuration** 88 | 89 | - **Trusted Network**: Ben’s PC and Allan’s PC are on the `trusted_network` and can communicate with the Samba server. 90 | 91 | - **Untrusted Network**: The Malicious User’s PC is on a different network (`untrusted_network`) and cannot directly access the Samba server or the trusted machines. 92 | 93 | ### **Security Testing** 94 | - The trusted machines (Ben and Allan) can access the Samba share. 95 | - The malicious user should not be able to access the shared folder. 96 | - You can simulate potential security vulnerabilities or test firewall/ACL rules between trusted and untrusted networks. 97 | 98 | --- 99 | 100 | ### **Troubleshooting** 101 | 102 | - **Containers Keep Stopping**: 103 | - Ensure the Dockerfiles for all containers have an appropriate command to keep them running (e.g., `tail -f /dev/null` or an interactive shell). 104 | - Check the logs of the containers for errors using: 105 | ```bash 106 | docker logs 107 | ``` 108 | 109 | - **Network Isolation Issues**: 110 | - Ensure that each container is on the correct network (`trusted_network` or `untrusted_network`) by inspecting the network configuration in the Docker Compose file. 111 | 112 | - **Can authenticate through smb client**: 113 | - You have to add user's to Samba password database: 114 | ```bash 115 | smbpasswd -a ben 116 | ``` 117 | - If necessary you should enable the user too: 118 | ```bash 119 | smbpasswd -e ben 120 | ``` 121 | 122 | --- 123 | 124 | ### **File and Directory Permissions on Samba Server** 125 | 126 | The Samba server has the following configuration: 127 | - **Share Name**: `eng` 128 | - **Access Control**: The share is only accessible by users in the `eng` group (Ben and Allan). 129 | - **Permissions**: 130 | - Files are created with `0660` permissions. 131 | - Directories are created with `2770` permissions and the `setgid` bit is set. 132 | - The share is not browseable, ensuring it is hidden from general listings. 133 | 134 | You can modify these settings by editing the `smb.conf` or `eng.conf` configuration files inside the container. 135 | 136 | --- 137 | 138 | ### **Docker Compose File (`docker-compose.yml`) Overview** 139 | 140 | ```yaml 141 | version: '3.9' 142 | 143 | services: 144 | smb-server: 145 | build: 146 | context: . 147 | dockerfile: Dockerfile 148 | container_name: smb-server 149 | networks: 150 | trusted_network: 151 | 152 | ben-pc: 153 | build: 154 | context: . 155 | dockerfile: Dockerfile.clients 156 | container_name: ben-pc 157 | networks: 158 | trusted_network: 159 | 160 | allan-pc: 161 | build: 162 | context: . 163 | dockerfile: Dockerfile.clients 164 | container_name: allan-pc 165 | networks: 166 | trusted_network: 167 | 168 | malicious-user-pc: 169 | build: 170 | context: . 171 | dockerfile: Dockerfile.clients 172 | container_name: malicious-user-pc 173 | networks: 174 | untrusted_network: 175 | 176 | networks: 177 | trusted_network: 178 | driver: bridge 179 | untrusted_network: 180 | driver: bridge 181 | ``` 182 | 183 | - **`smb-server`**: The Samba server container providing file shares. 184 | - **`ben-pc`, `allan-pc`**: Trusted client containers that have access to the `eng` share. 185 | - **`malicious-user-pc`**: A container simulating a malicious user on an untrusted network. 186 | 187 | ### **Conclusion** 188 | 189 | This lab setup simulates a network environment with a Samba server, trusted, and untrusted clients. You can use it to test Samba sharing, network isolation, and security settings like access control and group-based file sharing. 190 | 191 | --- -------------------------------------------------------------------------------- /software-installation/data/PXE_Boot.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/abdoufermat5/unix-and-linux-sysadmin-notes/10b44b43dff4235fd3e4e9755d6f60361093b1e9/software-installation/data/PXE_Boot.png -------------------------------------------------------------------------------- /software-installation/data/intel-uefi-pxe-boot-performance-analysis.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/abdoufermat5/unix-and-linux-sysadmin-notes/10b44b43dff4235fd3e4e9755d6f60361093b1e9/software-installation/data/intel-uefi-pxe-boot-performance-analysis.pdf -------------------------------------------------------------------------------- /software-installation/readme.md: -------------------------------------------------------------------------------- 1 | # Chapter 6: Software Installation and Management 2 | 3 | ## Operating System Installation 4 | 5 | Linux distributions and FreeBSD have straightforward procedures for basic installation. For physical machines, you can boot from a CD, DVD, or USB drive. For virtual machines, you can boot from an ISO file. Installing the base OS from local media is fairly trivial thanks to the GUI apps that shepherd you through the process. 6 | 7 | ### Installation from the network 8 | 9 | If you have to install OS on more than one computer, you'll quickly reach the limits of the local media approach. It's time consuming, error prone, and boring to repeat the same steps over and over. The solution is to install the OS from a network server. This is a common practice in data centers and cloud environments. 10 | 11 | The most common methods use DHCP and TFTP to boot the system sans physical media. They then retrieves the OS installation files from a network server with HTTP, FTP, or NFS. The installation files can be on the same server or on a different one. 12 | 13 | We can set up a completely hands-free installations through PXE, the Preboot eXecution Environment. This scheme is a standard from Intel that lets systems boot from network interface. 14 | 15 | PXE acts like a miniature OS that sits in a ROM on the network card. It exposes its network capabilities through a standardized API for the system BIOS to use. This cooperation makes it possible for a single boot loader to netboot any PXE-enabled PC without having to supply special drivers for each network card. 16 | 17 | ![PXE Boot](./data/PXE_Boot.png) 18 | 19 | ## Linux Package Management Systems 20 | 21 | Two package formats are in common use on Linux systems. Red Hat, CentOS, SUSE, Amazon Linux, and several other distributions use **RPM**. Debian and Ubuntu use the separate but equally popular **.deb** format. The two formats are functionally similar. 22 | 23 | Both the RPM and .deb packaging systems now function as dual-layer soup-to-nuts configuration management tools. At the lowest level are the tools that install, uninstall, and query packages: **rpm** for **RPM** and **dpkg** for **.deb.** 24 | 25 | On top of these commands are systems that know how to find and download packages from the Internet, analyze interpackage dependencies, and upgrade all the packages on a system. **yum**, the Yellowdog Updater, Modified, works with the **RPM** system. **APT**, the Advanced Package Tool, originated in the **.deb** universe but works well with both **.deb** and **RPM** packages. 26 | 27 | ## High-Level Package Management 28 | 29 | The high-level package management tools are the ones you'll use most often. They are the ones that let you install, remove, and upgrade packages. They also let you search for packages and list the packages installed on your system. 30 | 31 | ### Package repositories 32 | 33 | Linux distributors maintain software repositories that work hand-in-hand with their chosen package management systems. The default configuration for the package management system usually points to one or more well-known web or FTP servers that are under the distributor’s control. 34 | 35 | - A *release* is a self-consistent snapshot of the package universe. 36 | - A *component* is a subset of the software within a release. 37 | - An *architecture* represents a class of hardware. The expectation is that machines within an architecture class are similar enough that they can run the same binaries. Architectures are instances of releases, for example, the i386 architecture of the Fedora 20 release. 38 | 39 | ### APT: Advanced Package Tool 40 | 41 | APT is a set of tools for managing Debian packages. It's the most widely used package management system for Debian-based systems. APT is a collection of tools that work together to provide a complete package management system. The tools are: 42 | 43 | - **apt-get**: The command-line tool for handling packages. It performs package management tasks such as installation, removal, and upgrade. 44 | - **apt-cache**: A tool for searching and querying the APT package cache. 45 | - **apt-file**: A tool for searching for files within packages. 46 | - **apt-show-versions**: A tool for showing versions of packages. 47 | - **aptitude**: A high-level interface to the package management system. It can be used to perform most of the tasks that **apt-get** can do, and many more. 48 | - **apt-mirror**: This tool allows you to mirror a package repository. 49 | 50 | The first rule of using APT on ubuntu systems is to ignore the existence of **dselect**, which act as a frontend for the Debian package system. 51 | 52 | ### yum: Yellowdog Updater, Modified 53 | 54 | **yum** is a package manager for RPM-compatible Linux systems. It is a high-level tool for managing packages. Yum performs dependency resolution when installing, updating, and removing packages. It can manage packages from installed repositories, and it can also perform command-line operations on individual packages. 55 | 56 | ## Software localization and configuration 57 | 58 | Adapting systems to your local (or cloud) environment is one of the prime battlegrounds of system administration. Addressing localization issues in a structured and reproducible way helps avoid the creation of snowflake systems that are impossible to recover after a major incident. 59 | 60 | -------------------------------------------------------------------------------- /software-installation/training/pxe.md: -------------------------------------------------------------------------------- 1 | # Preboot eXecution Environment (PXE) 2 | 3 | Preboot Execution Environment (PXE) defines a method for booting computers using a network interface, independent of local storage devices or installed operating systems (OSs). On platforms with UEFI firmware, PXE is supported by a network stack in the client firmware. The network’s DHCP provides a path to a boot server and network bootstrap program (NBP), downloads it into the computer's local memory using TFTP, verifies the image, and executes the NBP. 4 | 5 | - In a Windows Deployment Services (WDS) environment, the NBP is provided by `wdsmgfw.efi`. 6 | - In a Linux environment, the NBP is provided by UEFI-enabled boot loaders such as GRUB, GRUB2 or ELILO. 7 | 8 | ## History 9 | 10 | PXE was introduced as part of the Wired for Management Baseline (WfM) Specification by Intel Corporation in 1997. It was described in a separate PXE 1.0 specification since Wired for Management 2.0. Later, the 2.1 update was published in September 1999. 11 | 12 | PXE 2.1 describes the IPv4-based network boot process. It does not cover IPv6-based PXE, but this is described in the UEFI 2.2 specification. The UEFI 2.6 specification describes the IPv6-based PXE process in Section 23.3.1. The DHCP6 options used in PXE process are also described in the UEFI specification. 13 | 14 | ## Related protocols 15 | 16 | The UEFI specification introduces the following protocols related to PXE boot: 17 | 18 | - PXE Base Code Protocol – provides several features to utilize the PXE-compatible devices, for network access and network booting. 19 | - PXE Base Code Callback Protocol – provides callback function which will be invoked when the PXE Base Code Protocol is about to transmit, has received, or is waiting to receive a packet. 20 | - Load File Protocol – loads the boot file to specified buffer, which allows the boot manager to boot the file later. 21 | 22 | ## PXE DHCP Timeout 23 | 24 | In IPv4-based PXE, DHCP discovery will be retried four times. The four timeouts are 4, 8, 16 and 32 seconds respectively, to compliant with PXE 2.1 specification. The initial retransmission timeout is 4 seconds and maximum retransmission timeout for each retry is 32 seconds. PXE client should wait for the timeout then select most preferred offer among all the received offers. 25 | 26 | ## PXE Boot process 27 | 28 | The following picture shows a typical IPv4 PXE 29 | ![BootProcess](../data/PXE_Boot.png) 30 | 31 | **Step 1-4** is DHCP protocol with several extended DHCP option tags. The client should broadcast a DHCP Discover message with "PXEClient" extension tags to trigger the DHCP process. Then it should select offers, get the address configuration and boot path information, and complete the standard DHCP protocol by sending a request for the selected address to the server and waiting for the Ack. It might also need to perform DNS resolution to translate the server's host address to IP address. 32 | 33 | **Step 5-6** takes place between the client and a Boot Server. The client should select and discover a Boot Server from the obtained server list in step 1-4. This phase is not a part of standard DHCP protocol, but uses the DHCP Request and Ack message format as a convenient for communication. The client should send the request message to port 67 (broadcast) or port 4011 (multicast/unicast) of the selected boot server, and wait a DHCP ack for the boot file name and MTFTP configuration parameters. 34 | 35 | **step 7-9** is the downloading of the network bootstrap program (NBP). The client will load the NBP into the computer’s local memory using TFTP, verify the image and execute it finally. 36 | 37 | - In a Windows Deployment Services (WDS) environment, the NBP is provided by `wdsmgfw.efi`. 38 | - In a Linux environment, the NBP is provided by UEFI-enabled boot loaders such as GRUB, GRUB2 or ELILO. 39 | 40 | Take UEFI PXE with Microsoft WDS as example, the NBP would continue to download several files to client platform. After the downloading finished, the WDS loader calls ExitBootService() and transits to Runtime phase. The OS kernel starts execution and takes over the control to the system. The OS network stack is also started for handling network operations. 41 | 42 | ## PXE Limitations 43 | 44 | PXE is a great tool for booting a large number of computers over a network. However, it has some limitations: 45 | 46 | - PXE uses UDP as transport protocol. TCP is not supported. 47 | - Router/Switch “fast learning spanning tree” may drop UDP packets. 48 | - PXE is designed to work within a corporate network, not outside of a company firewall. 49 | - PXE server must be on same subnet. 50 | - Requires Modifications to the DHCP Server 51 | - PXE uses TFTP qnd does not support a seccure transport method like HTTPS. 52 | 53 | ## Setting up PXE 54 | 55 | The most widely used PXE boot system is H. Peter Anvin's PXELINUX, which is part of hist SYSLINUX suite of boot loaders for every occasion. Another option is IPXE, which supports additional bootstrapping modes, including support for wireless networks. 56 | 57 | PXELINUX supplies a boot file that you install in the TFTP server's **tftpboot** directory. To boot from the network, a PC downloads the PXE boot loader and its configuration from the TFTP server. The configuration file lists one or more options for operating systems to boot. The system can boot through to a specific OS installation without any user intervention, or it can display a custom boot menu. 58 | 59 | PXELINUX uses the PXE API for its downloads and is therefore hardware independent all the way through the boot process. 60 | 61 | On the DHCP side, ISC's (the Internet Systems Consortium's) DHCP server is the most widely used for PXE information. Alternatively, there's Dnsmasq, a lightweight server with DNS, DHCP, and netboot support. Or we can simply use Cobbler. 62 | 63 | 64 | -------------------------------------------------------------------------------- /storage/data/cow.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/abdoufermat5/unix-and-linux-sysadmin-notes/10b44b43dff4235fd3e4e9755d6f60361093b1e9/storage/data/cow.png -------------------------------------------------------------------------------- /storage/data/fdisk-recipe.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/abdoufermat5/unix-and-linux-sysadmin-notes/10b44b43dff4235fd3e4e9755d6f60361093b1e9/storage/data/fdisk-recipe.png -------------------------------------------------------------------------------- /storage/data/fs-space-management.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/abdoufermat5/unix-and-linux-sysadmin-notes/10b44b43dff4235fd3e4e9755d6f60361093b1e9/storage/data/fs-space-management.png -------------------------------------------------------------------------------- /storage/data/hdd-vs-ssd.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/abdoufermat5/unix-and-linux-sysadmin-notes/10b44b43dff4235fd3e4e9755d6f60361093b1e9/storage/data/hdd-vs-ssd.png -------------------------------------------------------------------------------- /storage/data/logical-vol-capacities.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/abdoufermat5/unix-and-linux-sysadmin-notes/10b44b43dff4235fd3e4e9755d6f60361093b1e9/storage/data/logical-vol-capacities.png -------------------------------------------------------------------------------- /storage/data/lvm-commands-linux.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/abdoufermat5/unix-and-linux-sysadmin-notes/10b44b43dff4235fd3e4e9755d6f60361093b1e9/storage/data/lvm-commands-linux.png -------------------------------------------------------------------------------- /storage/data/raid-0.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/abdoufermat5/unix-and-linux-sysadmin-notes/10b44b43dff4235fd3e4e9755d6f60361093b1e9/storage/data/raid-0.png -------------------------------------------------------------------------------- /storage/data/raid-1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/abdoufermat5/unix-and-linux-sysadmin-notes/10b44b43dff4235fd3e4e9755d6f60361093b1e9/storage/data/raid-1.png -------------------------------------------------------------------------------- /storage/data/raid-10-01.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/abdoufermat5/unix-and-linux-sysadmin-notes/10b44b43dff4235fd3e4e9755d6f60361093b1e9/storage/data/raid-10-01.png -------------------------------------------------------------------------------- /storage/data/raid-5.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/abdoufermat5/unix-and-linux-sysadmin-notes/10b44b43dff4235fd3e4e9755d6f60361093b1e9/storage/data/raid-5.png -------------------------------------------------------------------------------- /storage/data/raid-6.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/abdoufermat5/unix-and-linux-sysadmin-notes/10b44b43dff4235fd3e4e9755d6f60361093b1e9/storage/data/raid-6.png -------------------------------------------------------------------------------- /storage/data/storage-mgmt-layer.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/abdoufermat5/unix-and-linux-sysadmin-notes/10b44b43dff4235fd3e4e9755d6f60361093b1e9/storage/data/storage-mgmt-layer.png -------------------------------------------------------------------------------- /storage/data/trad-part-scheme.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/abdoufermat5/unix-and-linux-sysadmin-notes/10b44b43dff4235fd3e4e9755d6f60361093b1e9/storage/data/trad-part-scheme.png -------------------------------------------------------------------------------- /storage/data/vgdisplay-out.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/abdoufermat5/unix-and-linux-sysadmin-notes/10b44b43dff4235fd3e4e9755d6f60361093b1e9/storage/data/vgdisplay-out.png -------------------------------------------------------------------------------- /storage/data/vgdisplay-out2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/abdoufermat5/unix-and-linux-sysadmin-notes/10b44b43dff4235fd3e4e9755d6f60361093b1e9/storage/data/vgdisplay-out2.png -------------------------------------------------------------------------------- /storage/data/zfs-archi.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/abdoufermat5/unix-and-linux-sysadmin-notes/10b44b43dff4235fd3e4e9755d6f60361093b1e9/storage/data/zfs-archi.png -------------------------------------------------------------------------------- /storage/training/YM_RAID_Primer.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/abdoufermat5/unix-and-linux-sysadmin-notes/10b44b43dff4235fd3e4e9755d6f60361093b1e9/storage/training/YM_RAID_Primer.pdf -------------------------------------------------------------------------------- /tcp-ip-networking/data/etc-hosts.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/abdoufermat5/unix-and-linux-sysadmin-notes/10b44b43dff4235fd3e4e9755d6f60361093b1e9/tcp-ip-networking/data/etc-hosts.png -------------------------------------------------------------------------------- /tcp-ip-networking/data/iptables-flags.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/abdoufermat5/unix-and-linux-sysadmin-notes/10b44b43dff4235fd3e4e9755d6f60361093b1e9/tcp-ip-networking/data/iptables-flags.png -------------------------------------------------------------------------------- /tcp-ip-networking/data/netmask.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/abdoufermat5/unix-and-linux-sysadmin-notes/10b44b43dff4235fd3e4e9755d6f60361093b1e9/tcp-ip-networking/data/netmask.png -------------------------------------------------------------------------------- /tcp-ip-networking/data/syn-ack.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/abdoufermat5/unix-and-linux-sysadmin-notes/10b44b43dff4235fd3e4e9755d6f60361093b1e9/tcp-ip-networking/data/syn-ack.png -------------------------------------------------------------------------------- /tcp-ip-networking/data/tcp-ip_layer_model.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/abdoufermat5/unix-and-linux-sysadmin-notes/10b44b43dff4235fd3e4e9755d6f60361093b1e9/tcp-ip-networking/data/tcp-ip_layer_model.png -------------------------------------------------------------------------------- /tcp-ip-networking/training/iptables.md: -------------------------------------------------------------------------------- 1 | # Iptables 2 | 3 | ![Iptables](https://data-flair.training/blogs/wp-content/uploads/sites/2/2022/04/iptables-in-linux.webp) 4 | 5 | ## Introduction 6 | 7 | The way the Firewall works is quite simple. It creates a barrier between trustworthy and untrustworthy networks so your system can be safe from malicious packets. 8 | 9 | IPTables can be used for personal computing or can also be applied to the entire network. Using IPTables, we will be defining a set of rules by which we can monitor, allow or block incoming or outgoing network packets. 10 | 11 | ## Understanding the concept of IPTables 12 | 13 | While discussing IPTables, we must understand 3 terms: **Tables**, **Chains**, and **Rules**. 14 | 15 | ### Tables 16 | 17 | Tables are the top-level structure in IPTables. There are 5 types of tables in IPTables and each has different rules applied. The tables are: 18 | 19 | 1. **Filter**: This is the default table in IPTables. It is used to filter packets based on the rules defined. 20 | 2. **NAT**: This table is used for Network Address Translation. It is used to translate the source or destination IP address of packets. 21 | 3. **Mangle**: This table is used to alter the IP packets. For example, it can change the TTL value of the packet. 22 | 4. **Raw**: This table is used to configure exemptions from connection tracking. 23 | 5. **Security**: This table is used to configure SELinux security policies. 24 | 25 | ### Chains 26 | 27 | Chains are the second-level structure in IPTables. Chains are used to define the rules for packets. There are 5 types of chains in IPTables: 28 | 29 | 1. **PREROUTING**: This chain is applied to any incoming packets before a routing decision is made regarding the final destination of the packet. 30 | 2. **INPUT**: It is the point where the packet is received by the network stack. 31 | 3. **FORWARD**: This chain is applied to packets that are being routed through the system. 32 | 4. **OUTPUT**: The output chain is applied to packets generated by the system and going out of the system. 33 | 5. **POSTROUTING**: This chain is applied to packets after they have been routed. 34 | 35 | ### Rules 36 | 37 | **Rules** are nothing but the set or individual commands by which users manipulate network traffic. Once each chain will come into action, the packet will be checked against defined rules. 38 | 39 | Each rule has two components: **Match** and **Target**. 40 | 41 | 1. **Match**: They are different conditions to define rules which can be matched by protocol, IP address, port, interface, header, etc. 42 | 2. **Target**: It is the action to be taken if the packet matches the rule. 43 | 44 | ## IPTables format 45 | 46 | The format of IPTables is as follows: 47 | 48 | ```bash 49 | iptables -t table -A chain -j target 50 | ``` 51 | 52 | ### Matching components 53 | 54 | The row named “[matching options]” is where you give a condition. If the condition is true, it will take the action, else it will move to the next rule in the chain. This detail provides the main function to filter the firewall. There is a huge list of parameters used for matching. But, broadly speaking, the parameters are divided into 3 types: generic parameters, implicit parameters, and explicit parameters. 55 | 56 | > a. Generic parameters: 57 | 58 | - **-p**: Protocol 59 | - **-s**: Source IP address 60 | - **-d**: Destination IP address 61 | - **-i**: Input interface 62 | - **-o**: Output interface 63 | 64 | ### Chain options 65 | 66 | - **-A**: Append a rule to the end of the chain. 67 | - **-C**: Checks for rule whether it satisfies the chain's requirements. 68 | - **-D**: Delete a rule from the chain. 69 | - **-I**: Insert a rule at the specified position in the chain. 70 | - **-F**: Flush the chain. 71 | - **-N**: Create a new chain. 72 | - **-X**: Delete a chain. 73 | 74 | ### Actions 75 | 76 | The row named “[target]” is where you give the action to be taken if the condition is true. The action can be one of the following: 77 | 78 | - **ACCEPT**: Accept the packet. 79 | - **DROP**: This target does not allow the connection, but send error message. 80 | - **REJECT**: This target blocks the connection. 81 | - **RETURN**: This target is used to stop the processing of the rules in the current chain for the packet. 82 | 83 | ## Examples 84 | 85 | ### 1. Allow SSH traffic 86 | 87 | ```bash 88 | iptables -A INPUT -p tcp --dport 22 -j ACCEPT 89 | ``` 90 | 91 | ### 2. Block SSH traffic 92 | 93 | ```bash 94 | iptables -A INPUT -p tcp --dport 22 -j DROP 95 | ``` 96 | 97 | ### 3. Block a specific IP address 98 | 99 | ```bash 100 | iptables -A INPUT -s 192.268.07.45 -j DROP 101 | ``` 102 | 103 | ### 4. Disable outgoing mails 104 | 105 | If you don’t want your system to send emails, you can block the SMTP ports 25, 465, and 587. 106 | 107 | ```bash 108 | iptables -A OUTPUT -p tcp --dport 25, 465, 587 -j REJECT 109 | ``` 110 | 111 | ### 5. Limit the number of concurrent connections 112 | 113 | If you have too many connections established from a single IP address on a given port (say SSH 22), you can limit the number of connections. 114 | 115 | ```bash 116 | iptables -A INPUT -p tcp --syn --dport 22 -m connlimit --connlimit-above 3 -j REJECT 117 | ``` 118 | 119 | The `connlimit` module is used to limit the number of parallel connections to a server per client IP address or per client IP address block. 120 | 121 | ### 6. Block ICMP traffic 122 | 123 | ```bash 124 | iptables -A INPUT -p icmp -j DROP 125 | ``` 126 | 127 | ### 7. Keep a log of dropped packets 128 | 129 | ```bash 130 | iptables -A INPUT -j LOG --log-prefix "Dropped: " 131 | ``` 132 | 133 | ### 8. Port forwarding 134 | 135 | If you want to forward the incoming traffic from port 80 to port 8080, you can use the following command: 136 | 137 | ```bash 138 | iptables -t nat -A PREROUTING -p tcp --dport 80 -j REDIRECT --to-port 8080 139 | ``` 140 | 141 | ### 9. Block outgoing traffic from my computer (interface eth0) to a specific website 142 | 143 | ```bash 144 | iptables -A OUTPUT -o eth0 -d www.example.com -j DROP 145 | ``` 146 | 147 | To allow it, you can use the following command: 148 | 149 | ```bash 150 | iptables -A OUTPUT -o eth0 -d www.example.com -j ACCEPT 151 | ``` -------------------------------------------------------------------------------- /the-filesystem/data/file-type-encoding.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/abdoufermat5/unix-and-linux-sysadmin-notes/10b44b43dff4235fd3e4e9755d6f60361093b1e9/the-filesystem/data/file-type-encoding.png -------------------------------------------------------------------------------- /the-filesystem/data/nsfv4.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/abdoufermat5/unix-and-linux-sysadmin-notes/10b44b43dff4235fd3e4e9755d6f60361093b1e9/the-filesystem/data/nsfv4.png -------------------------------------------------------------------------------- /the-filesystem/data/pathnames.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/abdoufermat5/unix-and-linux-sysadmin-notes/10b44b43dff4235fd3e4e9755d6f60361093b1e9/the-filesystem/data/pathnames.png -------------------------------------------------------------------------------- /the-filesystem/data/permissions-encoding.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/abdoufermat5/unix-and-linux-sysadmin-notes/10b44b43dff4235fd3e4e9755d6f60361093b1e9/the-filesystem/data/permissions-encoding.png -------------------------------------------------------------------------------- /the-filesystem/training/x-windows.md: -------------------------------------------------------------------------------- 1 | # X Windows System 2 | 3 | The X Window System (X11, or simply X) is a windowing system for bitmap displays, common on Unix-like operating systems. X provides the basic framework for a GUI environment: drawing and moving windows on the display device and interacting with a mouse and keyboard. X does not mandate the user interface – this is handled by individual programs. As such, the visual styling of X-based environments varies greatly; different programs may present radically different interfaces. 4 | 5 | ## X Window System Architecture 6 | 7 | ![x-windows-system](https://www.researchgate.net/publication/27516949/figure/fig3/AS:310001387229195@1450921498968/Simple-X-window-system-architecture.png) 8 | 9 | The X Window System uses a client–server model: an X server communicates with various client programs. The server accepts requests for graphical output (windows) and sends back user input (from keyboard, mouse, or touchscreen). 10 | 11 | -------------------------------------------------------------------------------- /update_read_status.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # This script is used to update the read status: Create the folder for the concerned chapter and create readme.md file in it. 4 | # Then update the base readme.md file by adding [Reading...] in front of the chapter name. 5 | # 6 | # Process: 7 | # Display the list of chapters in the base readme.md file. 8 | # Ask the user to enter the chapter number to update the read status. 9 | # Check if the chapter number is valid. 10 | # Check if the chapter is already read. 11 | # Create the folder for the concerned chapter and create readme.md file in it. 12 | # 13 | # Author: Abdoufermat 14 | # Date: 2020-02-27 15 | 16 | # Display the list of chapters in the base readme.md file. 17 | 18 | # Read the readme.md file and extract chapter information 19 | readme_file="README.md" 20 | chapter_pattern="^\- \[Chapter([0-9]+): (.+)\]\((.+)\)$" 21 | 22 | # Declare arrays to store chapter numbers, titles, and links 23 | declare -a chapter_numbers 24 | declare -a chapter_titles 25 | declare -a chapter_directories 26 | 27 | echo -e "List of chapters: \n" 28 | echo -e "--------------------------------------------------\n" 29 | # Read each line from the readme file 30 | while IFS= read -r line; do 31 | # Check if the line matches the chapter pattern 32 | if [[ $line =~ $chapter_pattern ]]; then 33 | chapter_numbers+=("${BASH_REMATCH[1]}") 34 | chapter_titles+=("${BASH_REMATCH[2]}") 35 | chapter_directories+=("${BASH_REMATCH[3]}") 36 | fi 37 | done <"$readme_file" 38 | 39 | # Display the list of chapters and associated directories in order 40 | for ((i = 0; i < ${#chapter_numbers[@]}; i++)); do 41 | chapter_number="${chapter_numbers[$i]}" 42 | chapter_title="${chapter_titles[$i]}" 43 | chapter_link="${chapter_directories[$i]}" 44 | 45 | echo "Chapter $chapter_number: $chapter_title" 46 | done 47 | 48 | echo -e "\n\n" 49 | 50 | # Ask the user to enter the chapter number to update the read status. 51 | read -p "Enter the chapter number to update the read status: " chapter_number 52 | 53 | # Check if the chapter number is valid. 54 | if [[ ! " ${chapter_numbers[@]} " =~ " ${chapter_number} " ]]; then 55 | echo "Invalid chapter number: $chapter_number\n" 56 | exit 1 57 | fi 58 | 59 | echo -e "\n\n" 60 | 61 | # Check if the chapter is already read (i.e., the directory exists) 62 | chapter_directory="${chapter_directories[$chapter_number - 1]}" 63 | if [ -d "$chapter_directory" ]; then 64 | echo "Chapter $chapter_number is already read or reading\n" 65 | exit 1 66 | fi 67 | 68 | echo -e "Chapter $chapter_number is not read yet\n\n Creating the folder for the concerned chapter and create readme.md file in it.\n...." 69 | # Create the folder for the concerned chapter and create readme.md file in it. 70 | mkdir -p "$(dirname "$chapter_directory")/data" 71 | mkdir -p "$(dirname "$chapter_directory")/training" 72 | echo -e "# Chapter $chapter_number: ${chapter_titles[$chapter_number - 1]}\n\n" >"$(dirname "$chapter_directory")/readme.md" 73 | 74 | echo -e "\n\nWe are set!!\n\nGood luck with Chapter $chapter_number!!\n" 75 | -------------------------------------------------------------------------------- /user-management/data/command-and-config.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/abdoufermat5/unix-and-linux-sysadmin-notes/10b44b43dff4235fd3e4e9755d6f60361093b1e9/user-management/data/command-and-config.png -------------------------------------------------------------------------------- /user-management/data/common-scripts.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/abdoufermat5/unix-and-linux-sysadmin-notes/10b44b43dff4235fd3e4e9755d6f60361093b1e9/user-management/data/common-scripts.png -------------------------------------------------------------------------------- /user-management/test: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/abdoufermat5/unix-and-linux-sysadmin-notes/10b44b43dff4235fd3e4e9755d6f60361093b1e9/user-management/test -------------------------------------------------------------------------------- /virtualization/data/containerization.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/abdoufermat5/unix-and-linux-sysadmin-notes/10b44b43dff4235fd3e4e9755d6f60361093b1e9/virtualization/data/containerization.png -------------------------------------------------------------------------------- /virtualization/data/type1-vs-type2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/abdoufermat5/unix-and-linux-sysadmin-notes/10b44b43dff4235fd3e4e9755d6f60361093b1e9/virtualization/data/type1-vs-type2.png -------------------------------------------------------------------------------- /virtualization/data/vm-vs-contd.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/abdoufermat5/unix-and-linux-sysadmin-notes/10b44b43dff4235fd3e4e9755d6f60361093b1e9/virtualization/data/vm-vs-contd.png -------------------------------------------------------------------------------- /virtualization/data/xen-comp-dom0.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/abdoufermat5/unix-and-linux-sysadmin-notes/10b44b43dff4235fd3e4e9755d6f60361093b1e9/virtualization/data/xen-comp-dom0.png -------------------------------------------------------------------------------- /virtualization/readme.md: -------------------------------------------------------------------------------- 1 | # Chapter 24: Virtualization 2 | 3 | ![vmware-meme](https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcRSQT0tgeMDlEh5wHKdL6FpVan8HP3vRTHWdw&s) 4 | 5 | Server virtualization makes it possible to run multiple operating system instances concurrently on the same physical hardware. Virtualization software parcels out CPU, memory, and I/O resources, dynamically allocating their use among several “guest” operating systems and resolving resource conflicts. From the user’s point of view, a virtual server walks and talks like a full-fledged physical server. 6 | 7 | The ever-growing size of server farms rekindled interest in virtualization for modern systems. VMware and other providers conquered the challenges of x86 and made it easy to automatically provision operating systems. These facilities eventually led to the rise of on-demand, Internet- connected virtual servers: the infrastructure we now know as cloud computing. More recently, advances in OS-level virtualization have ushered in a new era of OS abstraction in the form of containers. 8 | 9 | ## Virtual Vernacular 10 | 11 | ### Hypervisor 12 | 13 | A hypervisor (also known as a virtual machine monitor) is a software layer that mediates between virtual machines (VMs) and the underlying hardware on which they run. 14 | 15 | Hypervisors are responsible for sharing resources among the guest operating systems, which are isolated from one another and which access the hardware exclusively through the hyoervisor. 16 | 17 | Guest operating systems are independent, so they needn't be the same. CentOS, Ubuntu, and Windows can all run on the same hypervisor. VMware ESX, XenServer, and FreeBSD's bhyve are examples of hypervisors. The Linux kernel-based virtual machine (KVM) converts the Linux kernel into a hypervisor. 18 | 19 | **Full virtualization** 20 | 21 | The first hypervisors fully emulated the underlying hardware, defining virtual replacements for all the basic computing resources: hard disks, network devices, interrupts, motherboard hardware, BIOSs, and so on. This mode incurs a performance penalty because the hypervisor must translate every instruction from the guest operating system into a form that the host hardware can understand. 22 | 23 | Most hypervisors that offer full virtualization separate the task of maintaining multiple environments (virtualization) from the task of simulating the hardware within each environment (emulation). 24 | 25 | The most common emulation package used in these systems is an open source project called QEMU (Quick Emulator). 26 | 27 | **Paravirtualization** 28 | 29 | Paravirtualization is a technique that allows the guest operating system to communicate directly with the hypervisor to access hardware, rather than relying on the hypervisor to simulate hardware. This approach can be more efficient than full virtualization because it eliminates the need to translate instructions. However, it requires modifications to the guest operating system, which depends on the hypervisor's API. 30 | 31 | **Hardware-assisted virtualization** 32 | 33 | Intel and AMD introduced CPU features (Intel VT and AMD-V, respectively) that facilitate virtualization on the x86 platform. The hardware-assisted also known as "accelerated virtualization" virtualizes the CPU and memory controller, albeit under the control of the hypervisor. 34 | 35 | **Paravirtualized drivers** 36 | 37 | Paravirtualized drivers make hardware-assisted virtualization more efficient by bridging the gap between the hypervisor and guest OS without requiring major changes to the guest. They handle disk, network, and display tasks, while full virtualization is reserved for more obscure parts of the architecture (such as the BIOS or interrupt controller). 38 | 39 | **type 1 vs type 2 hypervisors** 40 | 41 | A type 1 hypervisor runs directly on the hardware without a supporting OS, and for that reasons is sometimes called a bare-metal or native hypervisor. Type 2 hypervisors are user-space applications that run on top of another general-purpose OS. 42 | 43 | ![type1-vs-type2](./data/type1-vs-type2.png) 44 | 45 | VMware ESXi and XenServer are considered type 1 hypervisors, and FreeBSD's bhyve is a type 2. VirtualBox and VMware Workstation are also type 2 hypervisors. 46 | 47 | ### Live migration 48 | 49 | Live migration is the process of moving a running virtual machine from one hypervisor to another without interrupting the VM's operation. This is useful for load balancing, hardware maintenance, and disaster recovery. The magic lies in a memory dance between the source and target hosts. 50 | 51 | In VMware ESXi, live migration is implemented using vMotion. 52 | 53 | ### Virtual machine images 54 | 55 | Virtual servers are created from images, which are templates of configured operating systems that hypervisors can load and execute. The format of these images varies by hypervisor. 56 | 57 | ### Containerization 58 | 59 | OS-level virtualization, also known as containerization, is a different approach that does not use a hypervisor. Instead, it relies on kernel features that isolate processes from the rest of the system. 60 | 61 | ![containerization](./data/containerization.png) 62 | 63 | Because it does not require virtualization of the hardware, the resource overhead of OS-level virtualization is low. Most implementations offer near-native performance. Linux LXC, Docker, and FreeBSD jails are examples of containerization technologies. 64 | 65 | It’s easy to confuse containers with virtual machines. Both define portable, isolated execution environments, and both look and act like full operating systems with root filesystems and running processes. Yet their implementations are entirely different. 66 | 67 | A true virtual machine has an OS kernel, an init process, drivers to interact with hardware, and the full trappings of a UNIX operating system. A container, on the other hand, is merely the facade of an OS (basically based on namespaces and cgroups). 68 | 69 | ![vm-vs-contd](./data/vm-vs-contd.png) 70 | 71 | ## Virtualization with Linux 72 | 73 | Xen and KVM are the leading open source virtualization projects for Linux. Xen is a type 1 hypervisor that runs on bare metal. It actually powers some Amazon's EC2 cloud or IBM's SoftLayer. 74 | 75 | ### Xen 76 | 77 | ![xen-arm-arch](https://wiki.xenproject.org/images/thumb/0/09/Xen_arch1.png/600px-Xen_arch1.png) 78 | ![xen-arch-2](https://wiki.xenproject.org/images/thumb/c/c1/Xen_arch2.png/630px-Xen_arch2.png) 79 | 80 | Initially developed at the University of Cambridge (by Ian Pratt), Xen is a bare-metal hypervisor that runs directly on the physical hardware. A running virtual machine is called a domain. There is always at least one domain, referred to as domain zero or `dom0`. Dom0 has full hardware access, manages the other domains, and runs all the hypervisor's own device drivers. Unprivileged domains are called `domU`. 81 | 82 | `Dom0` typically runs a Linux distribution. It looks just like any other Linux system but includes the daemons, tools, and libraries that complete the Xen architecture and enable communication among `domU`, `dom0`, and the hypervisor. 83 | 84 | The hypervisor is responsible for CPU scheduling and memory management for the system as a whole. It controls all domains, including `dom0`. However, the hypervisor itself is in turn controlled by `dom0`. 85 | 86 | ![confused-memes](https://uploads.dailydot.com/2024/04/confused-nick-young-.jpg?auto=compress&fm=pjpg) 87 | 88 | Each Xen guest-domain config file in `/etc/xen` specifies the virtual resources available to a `domU`, including disk devices, CPU, memory, and network interfaces. Each `domU` has a separate config file. 89 | 90 | ![xen-comp-dom0](./data/xen-comp-dom0.png) 91 | 92 | for more information on Xen, visit [Xen Project Wiki](https://wiki.xenproject.org/wiki/Xen_ARM_with_Virtualization_Extensions_whitepaper). 93 | 94 | ### Xen guest installation 95 | 96 | It takes several steps to get a guest server up and running under Xen. It's better to use a tool such as `virt-manager` through the `virt-install` command to create a new VM. `virt-install` is a command-line tool that accepts installation media from variety of sources, including SMB or NFS mounts, physical CDs or DVDs, and HTTP urls. 97 | 98 | Guest operating systems need a place to store their data (operating system files, user files, etc.), just like a physical computer needs a hard drive These disks are normally stored in virtual block devices (VBDs) in `dom0`. 99 | 100 | There are two main ways to implement VBDs: 101 | 102 | - Dedicated Resource: The VBD is directly linked to a physical disk or a logical volume (a partitioned section of a physical disk). This offers better performance because the guest has more direct access to the storage hardware. 103 | - Loopback File (File-Backed VBD): The VBD is stored as a regular file within dom0's filesystem. This file is a "sparse file," meaning it only uses disk space as the guest OS actually writes data, making it efficient in terms of storage usage. This approach is more flexible because you can manage the virtual disk using standard Linux commands (like `cp` or `mv`). 104 | 105 | Example of installation: 106 | 107 | ```bash 108 | virt-install -n chef -r 1024 -f /vm/chef.img -l http://example.com/myos --nographics 109 | ``` 110 | 111 | This command creates a new VM named `chef` with 1GB of RAM, a disk VBD at `/vm/chef.img`, and an installation source at `http://example.com/myos`. The `--nographics` flag tells `virt-install` to use a text-based installer. 112 | 113 | `virt-install` saves the domain's config in `/etc/xen/chef`: 114 | 115 | ```bash 116 | name = "chef" 117 | uuid = "f4e2e3b4-7f3d-4b1b-8b3b-3b4b1b7f3d4b" 118 | maxmem = 1024 119 | memory = 1024 120 | vcpus = 1 121 | bootloader = "/usr/bin/pygrub" 122 | on_poweroff = "destroy" 123 | on_reboot = "restart" 124 | on_crash = "restart" 125 | vfb = [ ] 126 | disk = [ "file:/vm/chef.dsk,xvda,w" ] 127 | vif = [ "mac=00:16:3e:2b:2b:2b,bridge=xenbr0" ] 128 | ``` 129 | 130 | The NIC is connected to `xenbr0`, a bridge device that connects the VM to the physical network. The writable disk image file is presented to the guest as `/dev/xvda`. 131 | 132 | To change the configuration of a guest domain (e.g., to attach another disk or to change the network to NAT mode instead of bridged mode), you can edit the config file directly and then reboot the guest. 133 | 134 | ### KVM 135 | 136 | KVM is not a clear case as it could be categorized as either type 1 or type 2. The KVM kernel module turns Linux kernel into a type 1 bare-metal hypervisor, while the overall system could be categorized to type 2 because the host OS is still fully functional and the other VM's are standard Linux processes from its perspective ([https://serverfault.com/a/855096](https://serverfault.com/a/855096)). 137 | 138 | Like Xen’s HVM mode, KVM takes advantage of the Intel VT and AMD-V CPU extensions and relies on QEMU to implement a fully virtualized hardware system. 139 | 140 | Under KVM, the Linux kernel itself serves as the hypervisor. Memory mamagement and scheduling are handled by the host kernel, and the guest OS runs as a normal process. 141 | 142 | ### KVM guest installation 143 | 144 | Although the technologies behind Xen and KVM are fundamentally different, the tools that install and manage guest operating systems are similar. As with Xen, you can use `virt-install` to create a new VM under KVM and then manage it with `virsh`. 145 | 146 | ```bash 147 | virt-install --connect qemu:///system -n UbuntuYakkety -r 512 -f ~/ubuntu-y.img -s 12 -c /dev/dvd --os-type linux --accelerate --hvm --vnc 148 | ``` 149 | 150 | This command creates a new VM named `UbuntuYakkety` with 512MB of RAM, a disk VBD at `~/ubuntu-y.img` (which can grow up to 12GB), a CD-ROM drive at `/dev/dvd`, and a VNC server for remote access. 151 | 152 | The `virsh` command-line tool can be used to manage KVM guests. It can be used to start, stop, pause, and resume VMs, as well as to query the status of a VM. It spawns its own shell, so you can run `virsh` commands interactively. 153 | 154 | ```bash 155 | sudo virsh --connect qemu:///system 156 | 157 | virsh # list --all 158 | Id Name State 159 | ---------------------------------------------------- 160 | 1 UbuntuYakkety running 161 | ``` 162 | 163 | ## FreeBSD's bhyve 164 | 165 | `bhyve` is a hypervisor that runs on FreeBSD. It is a type 2 hypervisor that relies on the host OS to manage memory and CPU scheduling. `bhyve` is a relatively new project, but it is gaining popularity because of its simplicity and performance. 166 | 167 | ## VMware 168 | 169 | VMware is the biggest player in the virtualization industry and was the first vendor to develop techniques to virtualize the fractious x86 platform. 170 | 171 | The primary product of interest to UNIX and Linux administrators is ESXi, which is a bare-metal hypervisor for the intel x86 architecture. The name ESXi stands for "Elastic Sky X Integrated." 172 | 173 | ![ok-this-is-crazy](https://i.imgflip.com/6qko6c.jpg) 174 | 175 | VMware have the most mature live migration technology, called `vMotion`. 176 | 177 | ## VirtualBox 178 | 179 | VirtualBox is a type 2 hypervisor that runs on Windows, macOS, and Linux. It is a popular choice for developers and hobbyists because it is free and easy to use. VirtualBox supports a wide range of guest operating systems, including Windows, Linux, and macOS. 180 | 181 | ## Packer 182 | 183 | Packer (packer.io), from the esteemed open source company HashiCorp, is a tool for building virtual machine images from a specification file. It can build images for a variety of virtualization and cloud platforms. Integrating Packer into your workflow lets you be more or less virtualization-platform-agnostic. You can easily build your customized image for whatever platform you’re using on a given day. 184 | 185 | This process is particularly helpful for supporting an “infrastructure as code” way of managing servers. Instead of manually applying changes to images, you modify a template that describes the image in abstract terms. Packer then builds the image for you. 186 | 187 | ## Vagrant 188 | 189 | Vagrant is a tool for managing virtual machines in a development environment. It is built on top of VirtualBox, VMware, and other virtualization technologies. Vagrant uses a simple text file called a Vagrantfile to describe the type of VM you want to create and how to configure it. You can use Vagrant to create a VM, provision it with software, and then destroy it when you’re done. 190 | 191 | -------------------------------------------------------------------------------- /virtualization/training/Namespaces_Cgroups_Containers.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/abdoufermat5/unix-and-linux-sysadmin-notes/10b44b43dff4235fd3e4e9755d6f60361093b1e9/virtualization/training/Namespaces_Cgroups_Containers.pdf -------------------------------------------------------------------------------- /virtualization/training/TorreyGuestLecture-Hypervors.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/abdoufermat5/unix-and-linux-sysadmin-notes/10b44b43dff4235fd3e4e9755d6f60361093b1e9/virtualization/training/TorreyGuestLecture-Hypervors.pdf -------------------------------------------------------------------------------- /web-hosting/data/apache-conf.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/abdoufermat5/unix-and-linux-sysadmin-notes/10b44b43dff4235fd3e4e9755d6f60361093b1e9/web-hosting/data/apache-conf.png -------------------------------------------------------------------------------- /web-hosting/data/cache-layers.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/abdoufermat5/unix-and-linux-sysadmin-notes/10b44b43dff4235fd3e4e9755d6f60361093b1e9/web-hosting/data/cache-layers.png -------------------------------------------------------------------------------- /web-hosting/data/cdn-work.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/abdoufermat5/unix-and-linux-sysadmin-notes/10b44b43dff4235fd3e4e9755d6f60361093b1e9/web-hosting/data/cdn-work.png -------------------------------------------------------------------------------- /web-hosting/data/common-headers.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/abdoufermat5/unix-and-linux-sysadmin-notes/10b44b43dff4235fd3e4e9755d6f60361093b1e9/web-hosting/data/common-headers.png -------------------------------------------------------------------------------- /web-hosting/data/components-web-stack.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/abdoufermat5/unix-and-linux-sysadmin-notes/10b44b43dff4235fd3e4e9755d6f60361093b1e9/web-hosting/data/components-web-stack.png -------------------------------------------------------------------------------- /web-hosting/data/ha-proxy.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/abdoufermat5/unix-and-linux-sysadmin-notes/10b44b43dff4235fd3e4e9755d6f60361093b1e9/web-hosting/data/ha-proxy.png -------------------------------------------------------------------------------- /web-hosting/data/http-response.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/abdoufermat5/unix-and-linux-sysadmin-notes/10b44b43dff4235fd3e4e9755d6f60361093b1e9/web-hosting/data/http-response.png -------------------------------------------------------------------------------- /web-hosting/data/http-server-types.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/abdoufermat5/unix-and-linux-sysadmin-notes/10b44b43dff4235fd3e4e9755d6f60361093b1e9/web-hosting/data/http-server-types.png -------------------------------------------------------------------------------- /web-hosting/data/load-balancer.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/abdoufermat5/unix-and-linux-sysadmin-notes/10b44b43dff4235fd3e4e9755d6f60361093b1e9/web-hosting/data/load-balancer.png -------------------------------------------------------------------------------- /web-hosting/data/nginx-conf-details-platform.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/abdoufermat5/unix-and-linux-sysadmin-notes/10b44b43dff4235fd3e4e9755d6f60361093b1e9/web-hosting/data/nginx-conf-details-platform.png -------------------------------------------------------------------------------- /web-hosting/data/open-source-caching.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/abdoufermat5/unix-and-linux-sysadmin-notes/10b44b43dff4235fd3e4e9755d6f60361093b1e9/web-hosting/data/open-source-caching.png -------------------------------------------------------------------------------- /web-hosting/data/subdirs-debian-apach.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/abdoufermat5/unix-and-linux-sysadmin-notes/10b44b43dff4235fd3e4e9755d6f60361093b1e9/web-hosting/data/subdirs-debian-apach.png -------------------------------------------------------------------------------- /web-hosting/training/all_about_load_balancing-wp-en.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/abdoufermat5/unix-and-linux-sysadmin-notes/10b44b43dff4235fd3e4e9755d6f60361093b1e9/web-hosting/training/all_about_load_balancing-wp-en.pdf -------------------------------------------------------------------------------- /web-hosting/training/reverse-proxy-cache.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/abdoufermat5/unix-and-linux-sysadmin-notes/10b44b43dff4235fd3e4e9755d6f60361093b1e9/web-hosting/training/reverse-proxy-cache.pdf -------------------------------------------------------------------------------- /where-to-start/data/other-sources.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/abdoufermat5/unix-and-linux-sysadmin-notes/10b44b43dff4235fd3e4e9755d6f60361093b1e9/where-to-start/data/other-sources.png -------------------------------------------------------------------------------- /where-to-start/data/table-of-linux-distros.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/abdoufermat5/unix-and-linux-sysadmin-notes/10b44b43dff4235fd3e4e9755d6f60361093b1e9/where-to-start/data/table-of-linux-distros.png -------------------------------------------------------------------------------- /where-to-start/readme.md: -------------------------------------------------------------------------------- 1 | # Chapter 1: Where to start 2 | 3 | ## Table of most popular linux distributions 4 | 5 | ![table](./data/table-of-linux-distros.png) 6 | 7 | The most viable distributions are not necessarily the most corporate. For example, we expect Debian GNU/Linux to remain viable for a long time despite the fact that Debian is not a company, doesn’t sell anything, and offers no enterprise-level support. Debian benefits from a committed group of contributors and from the enormous popularity of the Ubuntu distribution, which is based on it. 8 | 9 | ## Example of a Linux distribution 10 | 11 | > Debian (pronounced *deb-ian*, named after Debra and Ian Murdock) is one of the oldest and most well-regarded distributions. It is a noncommercial project with more than a thousand contributors worldwide. Debian maintains an ideological commitment to community development and open access, so there’s never any question about which parts of the distribution are free or redistributable. 12 | 13 | Debian defines three release that are maintained simultaneously: 14 | 15 | - stable: targeting the production servers, 16 | - unstable: with current packages that may have bugs and security vulnerabilities, 17 | - testing: a mix of stable and unstable. 18 | 19 | 20 | > Ubuntu: is based on Debian and maintains Debian's commitment to free and open-source software. Ubuntu is a commercial distribution, and it is backed by a company called Canonical. 21 | 22 | Ubuntu version numbers derive from the year and month of release, so 18.04 was released in April 2018. Each release also has a code name, such as Bionic Beaver for 18.04 or Focal Fossa for 20.04. 23 | 24 | Two versions of Ubuntu are released every year, one in April and one in October. The April release is a long-term support (LTS) release, which is supported for five years. The October release is supported for nine months. 25 | 26 | > Red Hat has been a dominant force in the Linux world for more than two decades, and its distributions are widely used in North America and beyond. By the numbers, Red Hat, Inc., is the most successful open source software company in the world. 27 | 28 | Red Hat Enterprise Linux, often shortened to RHEL, targets production environments at large enterprises that require support and consulting services to keep their systems running smoothly. Somewhat paradoxically, RHEL is open source but requires a license. If you’re not willing to pay for the license, you’re not going to be running Red Hat. 29 | 30 | Red Hat also sponsors Fedora, a community-driven distribution that is a proving ground for new technologies that may eventually be included in RHEL. Fedora is a good choice for developers and enthusiasts who want to stay on the cutting edge of Linux. 31 | s 32 | > CentOS is a free, open source, community-driven distribution that is functionally compatible with RHEL. The CentOS distribution lacks the RHEL branding and logos, but it is otherwise identical to RHEL. CentOS is a good choice for organizations that want the benefits of RHEL without the cost. 33 | 34 | > SUSE Linux Enterprise Server (SLES) is a commercial distribution that is popular in Europe. SLES is developed and maintained by the German company SUSE. SUSE also sponsors openSUSE, a community-driven distribution that is a proving ground for new technologies that may eventually be included in SLES. 35 | 36 | 37 | > FreeBSD, first release in late 1993, is the most widely used of the BSD derivatives. Unlike Linux, FreeBSD is a complete operating system, not just a kernel. Both the kernel and userland software are licensed under the permissive BSD License, a fact that encourages development by and additions from the business community. 38 | 39 | 40 | ## The man pages 41 | 42 | Man pages are concise descriptions of indidual commands, drivers; file formats, or library routines. THey do not address more general topics such as "How do I install a new device?" or "WHy is this system so damn slow?" 43 | 44 | On Linux systems, you can find out the current default search path with the *manpath* command. If necessary, you can set the MANPATH environment variable to override the default search path. 45 | 46 | ```bash 47 | $ export MANPATH=/home/share/localman:/usr/share/man 48 | ``` 49 | 50 | ## Other Sources 51 | 52 | - [Dark Reading](https://www.darkreading.com/) : Security news, research, and analysis. 53 | - [Devops Reactions](http://devopsreactions.tumblr.com/) : A collection of gifs that capture the feelings of sysadmins and developers. 54 | - [Linux](https://www.linux.com/) : The Linux Foundation's official website. 55 | - [Linux Foundation](https://www.linuxfoundation.org/) : Employer of Linus Torvalds and steward of the Linux kernel. 56 | - [LWN](https://lwn.net/) : A weekly publication that covers the Linux kernel and other open source software. 57 | - [Servers for hackers](https://serversforhackers.com/) : High-quality videos, forums, and articles on administration 58 | 59 | ## What is on my machine? 60 | 61 | ```bash 62 | $ which gcc 63 | /usr/bin/gcc 64 | ``` 65 | 66 | The *which* command searches the directories in your PATH environment variable for the specified command. If the command is found, the full path to the command is printed. If the command is not found, nothing is printed. 67 | 68 | There is also a *whereis* command that searches for the binary, source, and manual page files for a command. 69 | 70 | ```bash 71 | $ whereis gcc 72 | gcc: /usr/bin/gcc /usr/lib/gcc /usr/share/gcc /usr/share/man/man1/gcc.1.gz 73 | ``` 74 | 75 | If you are looking for a file, you can use the *locate* command. The *locate* command searches a database of files and directories on your system. The database is updated periodically by the *updatedb* command. 76 | 77 | ```bash 78 | $ locate my-unbelivable-script.sh 79 | /home/abdou/my-unbelivable-script.sh 80 | ``` 81 | 82 | ## Specialization and adjacent disciplines 83 | 84 | - **DevOps**: DevOps is not so much a specific function as a culture or operational philosophy. It aims to improve the efficiency of building and delivering software, especially at large sites that have many interrelated services and teams. Organizations with a DevOps practice promote integration among engineering teams and may draw little or no distinction between development and operations. Experts who work in this area seek out inefficient processes and replace them with small shell scripts or large and unwieldy Chef repositories. 85 | 86 | - **Site Reliability Engineering (SRE)**: Site reliability engineers value uptime and correctness above all else. Monitoring networks, deploying production software, taking pager duty, planning future expansion, and debugging outages all lie within the realm of these availability crusaders. Single points of failure are site reliability engineers’ nemeses. 87 | 88 | - **Architects**: Systems architects have deep expertise in more than one area. They use their experience to design distributed systems. Their job descriptions may include defining security zones and segmentation, eliminating single points of failure, planning for future growth, ensuring connectivity among multiple networks and third parties, and other site-wide decision making. Good architects are technically proficient and generally prefer to implement and test their own designs. --------------------------------------------------------------------------------