---
id: 2025-11-14-lpic-1-searching-and-extracting-data-from-files
slug: lpic-1-searching-and-extracting-data-from-files
title: "LPIC-1: Searching and extracting data from files"
excerpt: "Part 7 of the LPIC-1 series: find, locate, grep and regex to search files and extract text. Practical workflows, automation and tips for Linux admins."
date: "2025-11-14T22:02:32+01:00"
updated: "2025-11-14T23:20:00+01:00"
author:
name: "LΓ‘szlΓ³ KovΓ‘cs"
handle: "lkovacs"
category: "lpic-1-serie"
tags: ["lpic-1", "lpic-1-serie", "findgrep", "linuxtutorial", "regextutorial", "systemadmin", "systemadministration"]
reading_time: 25
toc: true
---
The first six [LPIC-1 modules](/en/category/lpic-1-serie){.badge-link-text} covered the Linux command line, filesystem navigation, viewing and editing file contents, and text processing with shell commands. [Streams, pipes and redirections](/en/lpic-1-serie/lpic-1-streams-pipes-and-redirections){.badge-link-text} then joined the input and output of different programs. The previous module covered [archiving and compression](/en/lpic-1-serie/lpic-1-archiving-and-compression){.badge-link-text}.
You can now display, change, forward and store data compactly. Day-to-day administration still needs one more step: you have to find the files and information you need first. On a small test system a few directory changes and a glance may still work. On a grown server with several filesystems, many configurations and large log trees, that approach reliably wastes time.
Part 7 of the series therefore covers targeted search for files and directories with `find`, `locate` and `plocate`. With `grep` you search file contents for fixed strings or regular expressions, while `sed` can process selected text further. The tools are then joined with pipes and `xargs`, picking up the concepts from the previous lessons.
The job is not only picking a matching command. Search scope, permissions, exit codes, safe filename handling and the limits of each tool matter just as much.
β **Important note:** This series does not replace an official exam-prep course for the LPIC-1 certification. It complements the exam topics with practical examples and technical context that help you learn and later operate Linux systems.
## Why searching and extracting is so central
Linux systems scatter information across many directories and files. Configurations live mainly under `/etc`, logs often under `/var/log`, application data under `/var/lib` and user-specific settings in the respective home directories. Package files, temporary data, backups and application-specific trees sit on top of that. Knowing that a piece of information *ought* to exist somewhere is rarely enough.
Administration searches on two levels. Either you need a particular file or directory, or you search inside files for a string or a pattern. The first job uses tools such as `find`, `locate` and `plocate`. File contents are searched with `grep` and regular expressions. `sed` can then select or change the matching text.
Typical tasks include:
* locating an unknown or moved configuration file,
* identifying large files when a filesystem is filling up,
* finding recently changed files after a broken update,
* checking files by owner or permissions,
* finding error messages in large log files,
* extracting specific values from configurations or command output,
* searching compressed logs without unpacking them permanently.
The tools complement each other. `find` searches the current state of the filesystem and can select hits by many properties. `locate` and `plocate` use a previously built index and usually return results faster, but they can contain stale entries. `grep` does not work with file properties; it works with the content of a file or a data stream.
The combination with [streams, pipes and redirections](/en/lpic-1-serie/lpic-1-streams-pipes-and-redirections){.badge-link-text} is what makes these commands important for administration. Search results can be handed to further programs, filtered or written to files. Several small tools then form a readable workflow without requiring a large script.
π‘ **Tip:** Before you search, decide whether you are looking for a path, a file property or a content match. That distinction chooses the tool and prevents pointless walks over large directory trees.
**A search is not automatically a harmless read.**
An unbounded `find /` can walk many mounted filesystems and generate substantial I/O load. Actions such as `-delete`, `-exec` or a follow-up with `xargs` can change or remove hits immediately. This module therefore keeps finding hits separate from the actions run on them.
β οΈ **Warning:** Inspect search results with a read-only listing before you delete, move or rewrite files. A syntactically correct search can still catch more files than you intended.
## Place in the LPIC-1 certification
Searching for files and filtering file contents is anchored in several exam objectives of LPIC-1 Exam 101. The basis is the currently published version 5.0 objectives for exam code 101-500.
The following exam objectives are particularly relevant here:
* **103.3 β Perform basic file management:** This objective covers using `find` to locate files by type, size or timestamps and then apply actions to the hits. The objective has weight 4.
* **103.7 β Search text files using regular expressions:** This includes basic and extended regular expressions and their use with `grep` and `sed`. Searching, deleting, changing and replacing text based on a pattern is also part of this objective. The weight is 3.
* **104.7 β Find system files and place files in the correct location:** This objective covers typical directories according to the Filesystem Hierarchy Standard and the tools `find`, `locate` and `updatedb`. It has weight 2.
The module also revisits material from earlier objectives. `sed` also belongs to processing text streams in **103.2**. Pipes and `xargs`, which you use to pass search results to further commands, are part of **103.4**.
The weight of an exam objective indicates its relative importance in the exam. A higher weight means more questions are likely. The weights do not translate into a fixed percentage for this module, because its content spans several objectives.
π‘ **Exam note:** Do not only memorise individual options. You should recognise whether a task is a search by file properties, a search of file contents, or a combination of tools. That boundary often decides whether `find`, `locate`, `grep` or a pipe with `xargs` is the right solution.
## What this module covers
The following sections introduce the main tools and concepts for targeted search of files, directories and contents on Linux:
* basics of file search and choosing the right tool
* `find` for name, type, size, timestamps, owners and permissions
* logical combinations and actions inside a `find` expression
* `locate` and `plocate` for fast searches over a path database
* `grep` for fixed strings and regular expressions
* basic and extended regular expressions
* selecting and changing text with `sed`
* searching compressed files and archives
* safe combinations of `find`, `grep`, pipes and `xargs`
* exercises for practice and exam preparation
The tools are not treated in isolation. You will see how they work together, where they stop, and how you inspect their results before further actions follow.
### Practical relevance for day-to-day admin work
Search is one of the most common jobs on a Linux system. Troubleshooting often starts not with a change, but with the question of where a relevant file lives and what information it contains.
**Typical uses include:**
* locating a service's configuration files,
* investigating error messages in current and older logs,
* identifying large or long-unchanged files,
* checking files with particular owners or permissions,
* narrowing down files changed after an update,
* comparing a given directive across several configuration files,
* extracting data from command output or text files,
* inspecting compressed logs and archives,
* passing search results safely to further commands.
The goal is not to build the longest possible command chain. A good search starts with a clearly limited scope and unambiguous criteria. The more precisely you know what you are looking for, the easier it is to judge results and avoid unintended actions.
As usual, the module uses practical markers for orientation:
* π‘ **Tips and notes** for a readable, efficient workflow
* β οΈ **Warnings and pitfalls** for costly or mutating commands
* π§ **Practical examples** to follow on the command line
* β **Typical failure modes** with a grounded explanation of the technical cause
### How to get the most from this module
Work the examples in a dedicated test environment. For most exercises a temporary directory in your home directory is enough. There you can create files with different names, contents, sizes and timestamps without touching production data.
Especially with `find`, regular expressions and `xargs`, reading a command is not enough. Small differences in quotes, parentheses or the order of expressions can change the result substantially. Run search commands that only list hits first. Add mutating actions only after you have checked the output.
β **Important note:** Do not use production directories for the exercises, and do not run mutating commands as `root`. A wrong search criterion stays wrong with administrative rights β it merely causes more thorough damage.
Watch three questions in every example:
Which scope is searched, which criteria select hits, and what happens to those hits afterwards? If you can answer those three points, you have understood the command and do not need to memorise it.
## Basics of data search on Linux
Two different jobs have to be separated before the individual tools: searching for files and directories, and searching for information inside files.
That distinction sounds obvious, but it decides which command is right. `find` can select a file by name, type or metadata. Whether that file contains a particular error message is something `find` does not know. For that you need a tool such as `grep`.
### Finding files vs searching file contents
When you search for files or directories, you work with properties of the filesystem entry. Those include, for example:
* name and path,
* file type,
* size,
* timestamps,
* owner and group,
* permissions.
A typical search goal is:
Find under `/var/log` all regular files that end in `.log` and were modified within the last seven days. The contents of the files are not inspected. Path, type, name and timestamp decide.
**A content search looks different:**
Find in the log files all lines that contain the word `error`. Now the files must be opened and their contents read. The filename may still matter, but it is not the actual search criterion. `grep` takes that job.
In practice both levels are often combined. First you select a limited set of files with `find`. Then you pass those files to `grep` to search their contents.
```markdown
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
β DECISION PATH: FILE VS. CONTENT SEARCH β
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ€
β β
β βββββββββββββββββββββββββ β
β β What do you want β β
β β to search for? β β
β βββββββββββββ¬ββββββββββββ β
β β β
β βββββββββββββββ΄ββββββββββββββ β
β βΌ βΌ β
β βββββββββββββββββββββ βββββββββββββββββββββ β
β β Files & paths β β Content in β β
β β (filesystem/meta) β β files (text) β β
β βββββββββββ¬ββββββββββ βββββββββββ¬ββββββββββ β
β β β β
β βΌ βΌ β
β βββββββββββββββββββββ βββββββββββββββββββββ β
β β find / locate β β grep / sed / awk β β
β βββββββββββ¬ββββββββββ βββββββββββββββββββββ β
β β β
β βββββββββββββββββββββββ β
β βΌ β
β βββββββββββββββββββββββββ β
β β Combine the tools β β
β β find + grep + xargs β β
β βββββββββββββββββββββββββ β
β β
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
```
The diagram shows the basic split: `find`, `locate` and `plocate` produce paths. `grep` inspects contents. `sed` can select or change matching text. Pipes, `-exec` or `xargs` join both levels.
### Choosing the right tool
The tools overlap in places, but they work on different principles:
| Tool | Searches | Basis | Typical use |
| --- | --- | --- | --- |
| `find` | files and directories | current filesystem state | precise search by name, type, size, time or permissions |
| `locate` | stored paths | previously built database | fast search for known file or directory names |
| `plocate` | stored paths | compressed search index | fast index-based path search on systems with `plocate` |
| `grep` | text files and streams | strings or regular expressions | search logs, configurations and command output |
| `sed` | text files and streams | addresses and regular expressions | select, replace or reshape text |
| `xargs` | no data of its own | input via `stdin` | turn input values into arguments for another command |
`find` and `locate` are therefore not a slower and a faster variant of the same command. `find` inspects the current state of a directory tree. `locate` and `plocate` read a database whose content depends on the last update.
`grep` and `find` do not replace each other either. `find` decides which files are selected. `grep` decides which lines or text fragments match a pattern.
π‘ **Tip:** Phrase the task as a sentence before you write the command. βFind files that β¦β usually leads to `find`. βFind lines that contain β¦β leads to `grep`. If the task contains both statements, you probably need a combination.
### Search scope, permissions and load
Every file search starts at a defined point. For `find` that is the given start path. The larger this scope, the more directories must be read and the more entries must be tested.
A search under `/etc` stays in a comparatively clear area. A search from `/` can reach local filesystems, attached disks, network mounts and virtual filesystems. Which areas are actually walked depends on the existing mount points and the options in use.
**Large search scopes have several effects:**
* The search takes longer and generates extra I/O load.
* Missing read permissions produce error messages or incomplete results.
* Mounted network filesystems can delay the search substantially.
* Mutating actions may catch more files than intended.
* Volatile directories can change while the search runs.
Not every permission error is a reason to run the whole command with `sudo`. First check whether the unreadable directories matter for your task at all. Administrative rights widen the search, but they also increase the possible damage of follow-up actions.
**Error messages can be hidden via standard error:**
```bash
find /var -name "*.conf" 2>/dev/null
```
Messages such as `Permission denied` then disappear from the display. The command does not gain extra rights. Unreadable directories are still not searched, and the output can be incomplete.
β οΈ **Warning:** Redirecting `stderr` to `/dev/null` does not fix an error. It only hides it. Use this redirection only when you can deliberately drop the messages and know that missing hits are acceptable for the task.
For large directory trees you should limit the search as early as possible. Suitable start paths, a controlled depth and staying on one filesystem reduce runtime and load. The corresponding `find` options are covered in the next section.
## find β search files and directories
`find` walks one or more directory trees and tests every entry it finds against an expression. That expression can contain tests, options, logical operators and actions. `find` is therefore not only a way to locate files, but also a tool for checks and controlled processing.
Unlike `locate`, `find` does not use a previously built index. The command reads the current filesystem state. New, moved or deleted files are therefore taken into account immediately, provided the calling user may search the directories in question.
### How it works and basic syntax
The basic syntax is:
```bash
find [start-path] [expression]
```
The start path sets where the search begins. The expression describes which entries match and what should happen to them.
A simple example:
```bash
find /etc -type f -name "*.conf"
```
The command starts under `/etc`, considers only regular files and selects names that end in `.conf`. Subdirectories are searched recursively by default.
A `find` expression consists of several elements:
* **Tests** check properties such as name, type, size or timestamps.
* **Options** affect the search itself, for example the maximum depth.
* **Operators** combine several conditions.
* **Actions** print hits or pass them to other commands.
If you specify no other action, `find` prints the paths of matching entries. In the simple case that output is the `-print` action.
```markdown
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
β FIND: RECURSIVE FILESYSTEM WALK β
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ€
β β
β βββββββββββββββββββ β
β β Start path β β
β β (e.g. /etc) β β
β ββββββββββ¬βββββββββ β
β β β
β βΌ β
β βββββββββββββββββββ β
β β Read directoriesβ β
β β recursively β β
β ββββββββββ¬βββββββββ β
β β β
β βΌ β
β βββββββββββββββββββ β
β β Test entry β β
β β against criteriaβ β
β ββββββββββ¬βββββββββ β
β β β
β ββββββββββ΄βββββββββ β
β βΌ βΌ β
β [ Match: Yes ] [ Match: No ] β
β β β β
β βΌ βΌ β
β Run the action Discard / β
β (-print / -exec) next entry β
β β
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
```
`find` evaluates the expression for every reachable entry. Which files are finally printed or processed therefore depends not only on the individual tests, but also on how they are combined.
### Search by name and type
With `-name` you search for the name of a directory entry. The pattern applies only to the last component of the path, not to the full path.
```bash
find /etc -name "sshd_config"
```
The same basic wildcards you already know from shell globbing are available:
* `*` stands for any number of characters,
* `?` stands for exactly one character,
* `[abc]` stands for one of the given characters.
All files with the `.conf` suffix, for example:
```bash
find /etc -name "*.conf"
```
The quotes matter. Without them the shell can expand the pattern before `find` starts. `find` then no longer receives the pattern, but possibly a list of existing filenames from the current directory.
β **Typical failure mode:** Put search patterns such as `"*.conf"` or `"log?.txt"` in quotes. Otherwise the shell may process the wildcard even though it was meant for `find`.
`-name` is case-sensitive. For a search that ignores case, GNU `find` provides `-iname`:
```bash
find /etc -iname "*.conf"
```
With `-path` you test the full path of the entry:
```bash
find /etc -path "*/systemd/*.conf"
```
To select a file type you use `-type`:
| Test | Meaning |
| --- | --- |
| `-type f` | regular file |
| `-type d` | directory |
| `-type l` | symbolic link |
| `-type b` | block device |
| `-type c` | character device |
| `-type p` | named pipe |
| `-type s` | socket |
A search for regular log files then looks like this:
```bash
find /var/log -type f -name "*.log"
```
Restricting to `-type f` prevents directories or other file types with the same name from appearing as hits.
### Search by size and timestamps
With `-size` you select files by their logical size:
```bash
find /var -type f -size +100M
```
The `+` means larger than the given value. A leading `-` means smaller than the value. Without a sign, `find` searches for the given number of units.
GNU `find` supports, among others, these units:
| Suffix | Unit |
| --- | --- |
| `c` | byte |
| `k` | kibibyte of 1024 bytes |
| `M` | mebibyte of 1024 Γ 1024 bytes |
| `G` | gibibyte of 1024 Γ 1024 Γ 1024 bytes |
| no suffix | 512-byte blocks |
The size is rounded up to whole units. `-size 1M` therefore does not mean the file must be exactly one mebibyte. It matches files whose size rounded up to whole mebibytes is one.
For empty regular files you can use `-empty`:
```bash
find /tmp -type f -empty
```
**Time-based tests distinguish several timestamps:**
* `-mtime` checks the last change of the file content,
* `-ctime` checks the last change of the inode metadata,
* `-atime` checks the last access,
* `-mmin`, `-cmin` and `-amin` work correspondingly in minutes.
Files whose content was changed within the last seven complete 24-hour periods:
```bash
find /var/log -type f -mtime -7
```
**The notation follows a fixed scheme:**
| Notation | Meaning |
| --- | --- |
| `-mtime 7` | seven complete 24-hour periods ago |
| `-mtime -7` | fewer than seven complete 24-hour periods ago |
| `-mtime +7` | more than seven complete 24-hour periods ago |
`find` rounds the span for `-mtime`, `-ctime` and `-atime` down to complete 24-hour periods. For tighter bounds the minute variants are better:
```bash
find /etc -type f -mmin -60
```
This command finds files whose content was changed fewer than 60 minutes ago.
With `-daystart`, GNU `find` computes day values from the beginning of the current calendar day instead of relative to the command start time:
```bash
find /var/log -daystart -type f -mtime 0
```
π‘ **Tip:** Use `-mmin` when the exact number of elapsed minutes matters. `-mtime` works with complete 24-hour periods and does not automatically mean a calendar day.
### Search by user, group and permissions
With `-user` and `-group` you select files by owner or group:
```bash
find /srv -type f -user www-data
```
```bash
find /srv -type f -group www-data
```
Numeric IDs use `-uid` and `-gid`. Files whose stored user or group ID no longer maps to a known entry on the system are found with `-nouser` and `-nogroup`:
```bash
find /home -xdev \( -nouser -o -nogroup \) -print
```
Such files can remain, for example, after a user account has been deleted.
The `-perm` test checks the permission bits. The exact notation is decisive:
```bash
find /srv -type f -perm 0644
```
This command finds only files whose permission bits are exactly `0644`. With a leading minus, all given bits must be set. Other bits may be present as well:
```bash
find /srv -type f -perm -0002
```
That finds regular files where write permission for others is set. Whether further rights exist does not matter for this test.
With a slash it is enough if at least one of the given bits is set:
```bash
find / -xdev -type f -perm /6000
```
This command finds regular files that have the SUID or SGID bit set.
| Notation | Check |
| --- | --- |
| `-perm 0644` | permissions are exactly `0644` |
| `-perm -0002` | all given bits are set |
| `-perm /6000` | at least one of the given bits is set |
β οΈ **Warning:** A search for striking permissions is only an inventory at first. Do not change rights wholesale with a follow-up `chmod`. Check first why the file has those permissions and which service depends on them.
### Logical AND, OR and NOT
Several tests are combined with a logical AND by default. The following command selects only entries that are both regular files and end in `.log`:
```bash
find /var/log -type f -name "*.log"
```
The spelled-out `-a` is possible, but usually not required:
```bash
find /var/log -type f -a -name "*.log"
```
For an OR combination you use `-o`:
```bash
find /var/log \( -name "*.log" -o -name "*.err" \)
```
The parentheses group the subexpression. They must be protected from the shell, for example with a backslash. If only regular files with one of the two suffixes should match, the full expression is:
```bash
find /var/log -type f \( -name "*.log" -o -name "*.err" \) -print
```
You negate a condition with `!`:
```bash
find /etc -type f ! -name "*.conf"
```
That finds regular files under `/etc` whose name does not end in `.conf`.
The operators have different precedence:
1. Parentheses group expressions explicitly.
2. `!` negates the following expression.
3. AND, or the implicit `-a`, is evaluated before OR.
4. `-o` forms the OR combination.
```markdown
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
β FIND: BOOLEAN LOGIC AND OPERATORS β
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ€
β β
β Expression: -type f AND ( -name "*.log" OR -name "*.err") β
β β β β
β β ββββββββ΄βββββββ β
β β βΌ βΌ β
β Regular Suffix Suffix β
β file .log .err β
β β β β β
β ββββββββββββββββΌββββββββββββββ β
β β β
β βΌ β
β Valid .log or .err file β
β β
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
```
Without parentheses, expressions can be evaluated differently from a first reading:
```bash
find /var/log -type f -name "*.log" -o -name "*.err"
```
That expression is equivalent to:
```markdown
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
β FIND: OPERATOR PRECEDENCE (AND BEFORE OR) β
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ€
β β
β Without parentheses -a (AND) binds tighter than -o (OR): β
β β
β -type f -name "*.log" -o -name "*.err" β
β βββββββββββββββββββββ βββββββββββββ β
β Group 1 Group 2 β
β (Regular logs only) OR (All *.err incl. dirs!) β
β β
β Correct with parentheses: β
β -type f \( -name "*.log" -o -name "*.err" \) β
β β
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
```
A directory with the `.err` suffix could therefore appear as a hit as well. Always group OR expressions explicitly when further conditions should apply to all alternatives.
β **Typical failure mode:** AND binds tighter than OR. If several name patterns should apply together with further conditions, the OR conditions belong in protected parentheses.
### Depth and filesystem boundaries
`find` searches subdirectories by default with no fixed depth limit. With `-maxdepth` you limit how far the command descends below the start path:
```bash
find /etc -maxdepth 2 -type f -name "*.conf"
```
The start path itself has depth 0. Entries contained directly in it sit at depth 1.
With `-mindepth` you set from which depth entries are tested:
```bash
find /tmp/testdata -mindepth 1 -maxdepth 1 -print
```
The start path `/tmp/testdata` itself is then not printed. Only its directly contained entries are considered.
The `-xdev` option prevents `find` from crossing into directories on other filesystems:
```bash
find / -xdev -type f -size +1G
```
The search stays on the filesystem of the start path. Other mounted filesystems must be given as their own start paths if needed.
`-prune` excludes particular directory trees from traversal. The following example skips `/var/cache`:
```bash
find /var -path /var/cache -prune -o -type f -name "*.log" -print
```
When `find` hits `/var/cache`, `-prune` prevents descent into that directory. For all other entries the right-hand side of the OR expression is evaluated.
π‘ **Tip:** Use as specific a start path as possible. `-xdev`, `-maxdepth` and `-prune` help you narrow the search, but they do not replace thinking first about which part of the filesystem is actually relevant.
Symbolic links are not followed by default. GNU `find` uses the behaviour of `-P` unless you specify otherwise. With `-L` the command follows symbolic links and tests the properties of their targets:
```bash
find -L /srv -type f -name "*.conf"
```
That can pull additional directory trees into the search. `-L` also changes what tests such as `-type` refer to.
β οΈ **Warning:** Use `-L` only deliberately. A symbolic link can point at a large or unexpected directory tree. Combined with mutating actions the consequences become hard to survey.
### Actions with `-print`, `-exec`, `-ok` and `-delete`
Tests decide which entries match. Actions decide what then happens to those hits.
The basic output action is `-print`:
```bash
find /etc -type f -name "*.conf" -print
```
`-ls` prints additional metadata:
```bash
find /etc -type f -name "*.conf" -ls
```
GNU `find` offers `-printf` for formatted output. The following example shows the size in bytes and the full path:
```bash
find /var/log -type f -printf "%s %p\n"
```
With `-exec` you pass found paths to another command. The token `{}` is replaced by the respective path.
```bash
find /etc -type f -name "*.conf" -exec file {} \;
```
The `\;` sequence ends the command to run. In this form `file` is invoked once for each hit.
The variant with `+` is often more efficient:
```bash
find /etc -type f -name "*.conf" -exec file {} +
```
Here `find` collects several paths and passes them together in as few invocations as possible. The argument lists are split so that the system-imposed maximum size is not exceeded.
```markdown
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
β FIND ACTIONS: -EXEC {} \; VS. -EXEC {} + β
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ€
β β
β One call per hit (-exec cmd {} \;) β
β Hit 1 βββΊ [ cmd Hit 1 ] β
β Hit 2 βββΊ [ cmd Hit 2 ] (N processes) β
β Hit 3 βββΊ [ cmd Hit 3 ] β
β β
β Batched call (-exec cmd {} +) β
β Hit 1 βββ β
β Hit 2 βββΌβββΊ [ cmd Hit 1 Hit 2 ... ] β
β Hit 3 βββ (1 process, minimal system load) β
β β
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
```
For commands that accept several filenames at once, `{} +` is usually the better choice. If each hit must be processed individually, `{} \;` remains required.
`-ok` works like `-exec`, but asks for confirmation before each invocation:
```bash
find /tmp/testdata -type f -name "*.tmp" -ok rm -- {} \;
```
That can help with a few hits, but it is not suitable for unattended scripts.
With `-delete`, GNU `find` deletes matching files and empty directories directly. Before deleting you should check exactly the same expression with `-print` first:
```bash
find /tmp/testdata -type f -name "*.tmp" -mtime +7 -print
```
Only when the output is correct do you replace the action:
```bash
find /tmp/testdata -type f -name "*.tmp" -mtime +7 -delete
```
`-delete` automatically enables depth-first traversal. Contents of a directory are therefore processed before the directory itself. That property makes the combination with `-prune` unsuitable, because `-prune` has no effect when `-depth` is active.
β οΈ **Warning:** `-delete` has no prompt and no recycle bin. Check start path, tests and the hit list immediately before deleting. Do not merely reuse a similar command from your shell history.
With `-exec` you should not splice paths into an unprotected shell string. This construction is fragile:
```bash
find /tmp/testdata -type f -exec sh -c "echo {}" \;
```
The found filename becomes part of the shell code. Special characters in the name can then be interpreted differently from what you intended. Pass paths as arguments instead:
```bash
find /tmp/testdata -type f -exec sh -c 'for path do printf "%s\n" "$path"; done' sh {} +
```
Inside the shell the found paths are then available as correctly separated arguments.
### Practical sysadmin examples
Practice scenario: find large files on a filesystem
```bash
find /var -xdev -type f -size +500M -print
```
The command searches under `/var` for regular files larger than 500 mebibytes. `-xdev` keeps it from entering other filesystems mounted underneath.
For output that can be sorted by size you can use `-printf`:
```bash
find /var -xdev -type f -size +500M -printf "%s %p\n" | sort -n
```
Practice scenario: find recently changed configurations
```bash
find /etc -type f -mmin -1440 -print
```
That finds regular files whose content was changed within the last 1440 minutes. After an update or a manual configuration change this can help you narrow the scope.
Practice scenario: find files without a known owner or group
```bash
find /home -xdev \( -nouser -o -nogroup \) -ls
```
The output shows files whose numeric user or group ID no longer maps to a current account or group.
Practice scenario: search configuration files for a directive
```bash
find /etc -type f -name "*.conf" -exec grep -HnF "Listen" {} +
```
`find` selects regular configuration files. `grep` searches their contents for the fixed string `Listen` and prints filenames and line numbers.
Practice scenario: list empty files in a temporary directory
```bash
find /tmp/testdata -type f -empty -print
```
Here too the command only prints. Whether empty files are actually faulty or disposable depends on the application.
β **Typical failure mode:** A hit is not proof that a file may be deleted or changed. `find` only checks the given criteria. It does not know the operational meaning of a file.
## locate and plocate β search via a file index
`locate` takes a different approach from `find`. Instead of walking the filesystem on every search, the command reads a previously built database. That database contains paths captured during an earlier scan.
`locate` is therefore well suited to fast searches for known file or directory names. The extra speed has a price: the result describes the state of the database, not necessarily the current filesystem.
On many current Linux systems the `locate` command is provided by `plocate`. Other systems still use `mlocate`, GNU `locate` or another implementation. The basic working model is similar, but individual options, default paths and the behaviour with several search patterns can differ.
Check which implementation is present:
```bash
command -v locate
locate --version
```
### How the database works
The database is created or updated with `updatedb`. `updatedb` walks the configured areas of the filesystem and stores the found paths in an index.
A later search with `locate` reads that index:
```markdown
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
β INDEX SEARCH: FILESYSTEM AND LOCATE DATABASE β
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ€
β β
β βββββββββββββββββββββββ β
β β Filesystem β Live state of all files β
β β (/etc, /usr, /var) β β
β ββββββββββββ¬βββββββββββ β
β β β
β β updatedb (indexing run) β
β βΌ β
β βββββββββββββββββββββββ β
β β Path database β Stored snapshot β
β β (/var/lib/mlocate) β β
β ββββββββββββ¬βββββββββββ β
β β β
β β locate pattern (extremely fast) β
β βΌ β
β βββββββββββββββββββββββ β
β β Paths found from β β
β β the index snapshot β β
β βββββββββββββββββββββββ β
β β
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
```
`locate` does not reopen every directory tree on a normal search. The command can therefore return results quickly even on systems with many files. The search is limited to the paths captured in the last `updatedb` run.
**A simple example:**
```bash
locate sshd_config
```
With `plocate`, a pattern without wildcards is treated as a substring by default. The command may therefore print `/etc/ssh/sshd_config` as well as backup copies or other paths that contain `sshd_config`.
π‘ **Tip:** Use `locate` for a quick orientation. Then check a found path with tools such as `ls`, `stat` or `test` when its current existence or properties matter for the next step.
### Updating with `updatedb`
On systems with `plocate` the system-wide database is usually refreshed regularly by a systemd timer. Other distributions or locate implementations may use a cron job or another mechanism.
Whether an automatic update is configured can be checked under systemd by searching the existing timers:
```bash
systemctl list-timers --all | grep -E "locate|updatedb"
```
If a newly created file should appear in the index immediately, the system-wide database can be updated manually:
```bash
sudo updatedb
```
Administrative rights are required for the system-wide database so that `updatedb` can read the configured directory trees and write the index at the configured location. An ordinary search with `locate` normally does not need root rights.
```markdown
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
β LOCATE DATABASE: FRESHNESS AND UPDATEDB β
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ€
β β
β Time 1: β
β Filesystem βββββ updatedb ββββββΊ Database current β
β β
β Time 2: β
β New file βββββββββββββββββββββββΊ Not yet in the index β
β Deleted file βββββββββββββββββββΊ Still listed wrongly β
β β
β Time 3: β
β Filesystem βββββ updatedb ββββββΊ Index in sync again β
β β
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
```
The location of the database depends on the implementation and the distribution. Paths such as `/var/lib/plocate/plocate.db` or `/var/lib/mlocate/mlocate.db` must therefore not be assumed as universal.
β οΈ **Warning:** Do not start `updatedb` unnecessarily in short intervals. Building or updating the index has to read directory structures and generates corresponding I/O load. On a correctly set up system a timer or cron job already does this.
### Configuring `updatedb`
With `plocate` the system-wide configuration is usually read from `/etc/updatedb.conf`. That file controls which filesystems, directories and paths are skipped when the database is built.
Important settings are:
| Setting | Meaning |
| --- | --- |
| `PRUNEFS` | filesystem types that are not searched |
| `PRUNENAMES` | directory names whose contents are not recorded |
| `PRUNEPATHS` | concrete directory paths whose contents are not recorded |
| `PRUNE_BIND_MOUNTS` | whether bind mounts are skipped |
The exact syntax and supported behaviour depend on the `updatedb` implementation. With `plocate`, `PRUNENAMES` contains directory names and not shell glob patterns. An entry such as `*.tmp` therefore does not generally exclude all temporary files.
Existing values should not be replaced unexamined with a shortened list of your own. Distributions often already list virtual filesystems, temporary areas or other unsuitable search targets there.
```bash
grep -Ev "^[[:space:]]*(#|$)" /etc/updatedb.conf
```
The command shows the effective, uncommented lines of the configuration. It does not change the file.
β **Typical failure mode:** Changes to `/etc/updatedb.conf` only affect an index built afterwards. A changed exclusion list does not rewrite an existing database retroactively.
### Patterns and important options
The following examples follow `plocate`. With another implementation check the local man page with `man locate`.
**A simple substring search:**
```bash
locate sshd_config
```
Ignore case with `-i`:
```bash
locate -i readme.md
```
With `-b` or `--basename` the pattern is compared only with the last component of the path:
```bash
locate -b sshd_config
```
`-b` does not force an exact match. A basename such as `sshd_config.backup` can still match the pattern.
With `-e` or `--existing`, `plocate` checks whether a database entry still exists in the filesystem at output time:
```bash
locate -e sshd_config
```
This check can suppress stale hits after deleted or moved files. It only adds an existence test and does not update the database.
Limit the number of printed hits with `-l`:
```bash
locate -l 20 "*.service"
```
The pattern is quoted so that the shell does not expand it.
With `-c`, `plocate` prints only the number of matching database entries:
```bash
locate -c "*.conf"
```
For regular expressions `plocate` offers `--regex` among other options. A full path that ends in `/sshd_config` can be searched like this:
```bash
locate --regex "/sshd_config$"
```
**Important options at a glance:**
| Option | Meaning |
| --- | --- |
| `-i`, `--ignore-case` | ignore case |
| `-b`, `--basename` | test only the basename |
| `-w`, `--wholename` | test the full path; default for `plocate` |
| `-e`, `--existing` | print only paths that still exist |
| `-l N`, `--limit N` | limit output to `N` hits |
| `-c`, `--count` | print only the number of hits |
| `-0`, `--null` | separate hits with a NUL character instead of a newline |
| `-S`, `--statistics` | show information about the database in use |
`-S` means statistics in `plocate` and GNU `locate`. The option does not sort search results:
```bash
locate -S
```
Format and content of the output are implementation-dependent.
β **Typical failure mode:** `locate` is not case-insensitive by default. If the spelling is unknown, you must give `-i` explicitly.
### Stale hits and implementation differences
**Between two updates three kinds of mismatch can occur:**
* A newly created file is still missing from the database.
* A deleted file is still in the index.
* A moved or renamed file appears under its old path and is missing under the new one.
With `plocate`, `-e` reduces stale hits by checking whether the path still exists before printing:
```bash
locate -e -b sshd_config
```
If you must inspect the current filesystem state completely, `find` remains the right tool:
```bash
find /etc -type f -name "sshd_config"
```
Behaviour with several search patterns is also not identical across implementations. `plocate` requires by default that a path matches all given patterns. Other locate variants can treat several patterns as alternatives. Portable scripts should therefore not rely on unexamined implementation-specific behaviour.
The database can also omit directories deliberately. A missing hit therefore does not automatically mean the file does not exist on the system. Its filesystem or directory may have been excluded when the index was built.
### Visibility and permissions
A system-wide path database contains information about many files. Modern implementations such as `plocate` therefore consider visibility for the calling user when the database is set up correctly. A path should only be printed if the user may read or enter the required parent directories.
This behaviour depends on ownership, permissions and the settings used when the database was built. A self-made database with overly broad read rights can disclose filenames that other users would not normally see.
β οΈ **Warning:** Do not treat a locate database as a harmless cache file. It contains an overview of stored paths and can therefore disclose confidential filenames or directory structures. Keep restrictive permissions on databases you create yourself.
A private database for a limited directory tree can be created with `plocate` like this:
```bash
updatedb -l 0 -U "$HOME/projects" -o "$HOME/projects.db"
```
**The database is then named explicitly for the search:**
```bash
locate -d "$HOME/projects.db" pattern
```
Every user who can read this database can recover the stored paths from it. The file should therefore not be world-readable:
```bash
chmod 600 "$HOME/projects.db"
```
### `locate` vs `find`
| Property | `locate` or `plocate` | `find` |
| --- | --- | --- |
| Data source | previously built path database | current filesystem |
| Freshness | depends on the last `updatedb` run | state during the search |
| Criteria | mainly path and name patterns | name, type, size, times, rights and further metadata |
| Speed | usually very fast with suitable patterns | depends on size and structure of the search scope |
| Unindexed areas | are not found | can be searched with sufficient permissions |
| Actions on hits | output for further processing | built-in actions such as `-exec` and `-delete` |
| Typical use | quick orientation by a known name | precise, current filesystem search |
Both tools therefore have a clear place. `locate` quickly answers where a given name occurs according to the index. `find` checks which entries actually exist in the given scope now and match the required properties.
π§ **Practical example:**
You are looking for the SSH server configuration file. For a quick orientation you first use:
```bash
locate -e -b sshd_config
```
If the search returns nothing, or you need a current finding limited to `/etc`, you inspect the filesystem directly:
```bash
find /etc -type f -name "sshd_config" -print
```
π‘ **Tip:** Start with `locate` when you know a name and want possible paths quickly. Use `find` when freshness, full control of the search scope or additional file properties are decisive.
Further details are in the local man pages with `man locate`, `man updatedb` and `man updatedb.conf`, and in the [plocate man pages](https://plocate.sesse.net/){.badge-link-text}.
## grep β search file contents
`grep` searches text files or data streams line by line for a pattern. If the pattern matches part of a line, `grep` prints the full line by default.
That is fundamentally different from `find`: `find` selects files by their properties, while `grep` inspects the contents of those files. Both tools can be combined later, but they should be understood separately first.
### Basic syntax and simple patterns
The basic syntax is:
```bash
grep [options] pattern [file...]
```
A simple search in a file:
```bash
grep "error" application.log
```
The command prints every line from `application.log` that matches the string `error`. Case is distinguished. `Error` or `ERROR` are therefore not hits.
Several files can be searched together:
```bash
grep "error" application.log system.log
```
When several files are given, `grep` prefixes the filename to each printed line by default. Without a file argument, `grep` reads from standard input.
**The command can therefore be used directly in a pipe:**
```bash
journalctl -u ssh.service | grep "Failed"
```
`journalctl` writes its output to `stdout`. The pipe passes that stream to `grep`, which prints only matching lines.
```markdown
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
β GREP: LINE-BY-LINE PATTERN CHECK AND OUTPUT β
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ€
β β
β βββββββββββββββββββββ β
β β File or stream β β
β βββββββββββ¬ββββββββββ β
β β β
β βΌ β
β βββββββββββββββββββββ β
β β grep tests line β β
β β by line (regex) β β
β βββββββββββ¬ββββββββββ β
β β β
β βββββββββββ΄ββββββββββ β
β βΌ βΌ β
β [ Match: Yes ] [ Match: No ] β
β β β β
β βΌ βΌ β
β Line on stdout No output β
β (filter/pipe) (ignored) β
β β
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
```
The pattern should be quoted. That stops spaces or characters that are meaningful to the shell from being processed before `grep` starts.
If the search pattern starts with a hyphen, `grep` can confuse it with an option. `--` ends option parsing:
```bash
grep -F -- "-Xms" jvm.options
```
### Fixed strings with `grep -F`
`grep` interprets its search pattern as a regular expression by default. Characters such as `.`, `*`, `[` or `^` can therefore have a special meaning.
When you search for a fixed string, use `-F` or `--fixed-strings`:
```bash
grep -F "server.name=web01.example.net" application.conf
```
With `-F` the dot is treated as an ordinary character. Without `-F` a dot in a regular expression stands for any character.
Another example:
```bash
grep -F "[error]" application.log
```
With `-F`, `grep` actually searches for the characters `[error]`. Without that option the expression would be interpreted as a character class.
π‘ **Tip:** Use `grep -F` when you search for a known word, path, configuration value or a complete error message and do not need regular expressions. That makes the intent of the command unambiguous and prevents surprising hits from metacharacters.
Several fixed strings can be given with several `-e` options:
```bash
grep -F -e "ERROR" -e "WARNING" application.log
```
The line is selected if at least one of the patterns matches.
**Patterns can also be read from a file:**
```bash
grep -F -f patterns.txt application.log
```
`grep` then interprets each line from `patterns.txt` as its own search pattern.
### Case, line numbers and context
With `-i`, `grep` ignores case:
```bash
grep -i "error" application.log
```
That matches `error`, `Error` and `ERROR`, among others.
`-n` adds the line number:
```bash
grep -nF "PermitRootLogin" /etc/ssh/sshd_config
```
**A possible output looks like this:**
```bash
33:PermitRootLogin no
```
With `-H` you force the filename to be printed:
```bash
grep -HnF "PermitRootLogin" /etc/ssh/sshd_config
```
`-h` suppresses the filename when several files are searched:
```bash
grep -hF "ERROR" app1.log app2.log
```
A single matching line is often not enough to judge a log entry. Surrounding lines use these options:
| Option | Meaning |
| --- | --- |
| `-A N` | print `N` lines after the hit |
| `-B N` | print `N` lines before the hit |
| `-C N` | print `N` lines before and after the hit |
Three lines before and after an error message:
```bash
grep -C 3 "connection refused" application.log
```
Only the following five lines:
```bash
grep -A 5 "Traceback" application.log
```
When several hit regions are separated, GNU `grep` marks the split with `--` by default.
π‘ **Tip:** Context lines are often more informative in logs than an isolated error message. Check in particular which messages were printed immediately before the error. The actual cause often sits there, while the last line only documents the collapse.
### Distinguishing matching lines, text fragments and filenames
Without special output options, `grep` prints full matching lines. Several options change what appears as the result.
With `-o` or `--only-matching`, GNU `grep` prints only the matching part of a line:
```bash
grep -oF "ERROR" application.log
```
If a line contains the pattern several times, `-o` can produce several outputs for the same input line.
`-c` counts matching lines:
```bash
grep -cF "ERROR" application.log
```
The printed value is not necessarily the number of all occurrences. If a line contains the word `ERROR` three times, `grep -c` still counts that line only once.
With `-l`, `grep` prints only the names of files that contain at least one matching line:
```bash
grep -lF "PermitRootLogin" /etc/ssh/sshd_config /etc/ssh/ssh_config
```
`-L` correspondingly shows only files without a matching content:
```bash
grep -LF "Managed by Ansible" /etc/*.conf
```
With `-q`, `grep` suppresses normal output completely:
```bash
grep -qF "PermitRootLogin no" /etc/ssh/sshd_config
```
This option is intended for conditions in scripts. The result is evaluated via the exit code.
**The main output options at a glance:**
| Option | Output |
| --- | --- |
| `-n` | matching lines with line number |
| `-H` | filename before each matching line |
| `-h` | suppress filenames |
| `-o` | only matching parts of a line |
| `-c` | number of matching lines |
| `-l` | names of files with hits |
| `-L` | names of files without hits |
| `-q` | no normal output |
β **Typical failure mode:** `grep -c` counts selected lines, not every individual match in general. If you want to count occurrences, you must first print the matching fragments with `-o` and then count them.
**A simple example for the number of individual occurrences:**
```bash
grep -oF "ERROR" application.log | wc -l
```
`wc -l` then counts the match lines printed by `grep -o`.
### Excluding hits
With `-v` you invert the selection. `grep` then prints every line that does *not* match the pattern:
```bash
grep -vF "DEBUG" application.log
```
**Several exclusions can be combined:**
```bash
grep -vF -e "DEBUG" -e "TRACE" application.log
```
That is useful, for example, when a log is mostly debug messages and you first want to look at the remaining entries.
Empty lines can be excluded with a regular expression:
```bash
grep -v "^$" application.conf
```
The exact meaning of `^` and `$` is covered in the section on regular expressions.
**Comments and empty lines can be hidden together:**
```bash
grep -Ev "^[[:space:]]*(#|$)" application.conf
```
This expression already uses extended regular expressions. It is fully broken down in the next main section.
### Searching multiple files and directories
With `-r`, GNU `grep` searches directories recursively:
```bash
grep -rF "PermitRootLogin" /etc/ssh
```
With GNU `grep`, `-r` follows symbolic links only when they were given explicitly as arguments on the command line. `-R` also follows symbolic links found during the recursive search.
```bash
grep -RF "PermitRootLogin" /etc/ssh
```
`-R` can therefore reach further or unexpectedly large directory trees.
β οΈ **Warning:** Use `grep -R` only when symbolic links should be followed deliberately. A link can point at another mount point, a large data set or back into an area already searched.
With `--include` you limit the recursive search to matching filenames:
```bash
grep -rF --include="*.conf" "PermitRootLogin" /etc
```
You exclude particular filenames with `--exclude`:
```bash
grep -rF --exclude="*.bak" "PermitRootLogin" /etc
```
Directories can be skipped with `--exclude-dir`:
```bash
grep -rF --exclude-dir=".git" "example.net" "$HOME/projects"
```
**Several patterns can be given repeatedly:**
```bash
grep -rF \
--include="*.conf" \
--exclude="*.bak" \
--exclude-dir=".git" \
"example.net" \
"$HOME/projects"
```
For more complex selection criteria a combination of `find` and `grep` is often clearer. `grep -r` is mainly suitable when selection by directory and filename is enough.
### Binary files and unusual input
If GNU `grep` determines that a file contains binary data, it does not necessarily print the matching lines by default. Instead a message such as this can appear:
```bash
Binary file firmware.bin matches
```
With `-I`, `grep` treats binary files as if they contained no hits:
```bash
grep -rIF "example.net" /opt/application
```
With `-a` or `--text` you force `grep` to treat a binary file as text:
```bash
grep -aF "example.net" firmware.bin
```
That can help with certain file formats, but it has risks. Binary data can contain control characters and disturb the terminal output.
β οΈ **Warning:** Do not casually pipe unchecked binary data with `grep -a` into an interactive terminal. For unknown files first use `file` to determine the type.
```bash
file firmware.bin
```
Compressed files are not ordinary text files either. Plain `grep` does not decompress them automatically. The dedicated tools such as `zgrep`, `bzgrep` and `xzgrep` are covered in a later section.
### Exit codes and use in scripts
`grep` distinguishes three basic results via its exit code:
| Exit code | Meaning |
| --- | --- |
| `0` | at least one line was selected |
| `1` | no line was selected |
| `2` | an error occurred |
A missing hit is therefore not a technical failure. That distinction matters for scripts.
```bash
grep -qF "PermitRootLogin no" /etc/ssh/sshd_config
status=$?
case "$status" in
0)
printf '%s\n' "Directive found"
;;
1)
printf '%s\n' "Directive not found"
;;
2)
printf '%s\n' "File could not be searched correctly" >&2
;;
esac
```
The `-q` option suppresses output, but it does not change the basic meaning of the exit codes. One peculiarity remains: once GNU `grep -q` has found a hit, it can exit with `0` even if accessing a further input file fails. If every input error must be detected reliably, `-q` should therefore not be used uncritically over many files.
After a pipe, `$?` in the shell contains only the exit code of the last command by default. Reliable evaluation of multi-stage pipelines is covered later with the combined workflows.
β **Typical failure mode:** An exit code of `1` does not mean `grep` failed. It only means that no matching line was found. A technical error is reported with `2`.
### Practical examples
Use case: show an SSH directive with line number
```bash
grep -HnF "PermitRootLogin" /etc/ssh/sshd_config
```
Use case: search for errors regardless of spelling
```bash
grep -inF "error" application.log
```
Use case: search only log files recursively
```bash
grep -rHnF --include="*.log" "connection refused" /var/log
```
Use case: hide comments and empty lines of a configuration
```bash
grep -Ev "^[[:space:]]*(#|$)" application.conf
```
Use case: find files that contain a given configuration directive
```bash
grep -rlF --include="*.conf" "server_name" /etc
```
Use case: check whether an error code occurs
```bash
if grep -qF "AH01630" apache-error.log; then
printf '%s\n' "Error code AH01630 was found"
else
status=$?
if [ "$status" -eq 1 ]; then
printf '%s\n' "Error code not found"
else
printf '%s\n' "Log file could not be read" >&2
fi
fi
```
π‘ **Tip:** Start with `grep -F` when you search for a concrete string. Switch to regular expressions only when the pattern actually has to be variable. A complicated expression is not a quality mark. Usually it is only complicated.
## Regular expressions with `grep` and `sed`
A fixed string describes concrete text. A regular expression describes a set of possible strings. You can therefore select lines that start with a date, contain a numeric error code or match several allowed spellings.
Regular expressions are not a programming language of their own and understand no operational meaning. A pattern can recognise that a string is shaped like an IPv4 address. Whether the individual values actually lie between 0 and 255 does not follow from that.
### Basic and extended regular expressions
POSIX distinguishes two basic syntax variants:
* **Basic Regular Expressions (BRE)** is what `grep` uses by default.
* **Extended Regular Expressions (ERE)** you enable with `grep -E`.
Both variants can describe essentially the same patterns under GNU `grep`. The main difference is the spelling of certain metacharacters.
| Function | BRE | ERE |
| --- | --- | --- |
| grouping | `\( β¦ \)` | `( β¦ )` |
| alternative | `\|` | `|` |
| zero or one | `\?` | `?` |
| one or more | `\+` | `+` |
| repetition range | `\{n,m\}` | `{n,m}` |
A pattern for lines that start with `ERROR` or `WARNING` is comparatively noisy in BRE:
```bash
grep '^\(ERROR\|WARNING\):' application.log
```
With ERE the same expression is more readable:
```bash
grep -E '^(ERROR|WARNING):' application.log
```
For new commands `grep -E` is usually the clearer choice as soon as grouping, alternatives or repetition ranges are needed. The older command `egrep` should no longer be used for that.
π‘ **Tip:** Choose BRE or ERE deliberately and stay with that syntax inside one expression. Many supposedly complicated regex errors only happen because parentheses or plus signs were escaped for the wrong variant.
### Literals, metacharacters and anchors
Most characters in a regular expression stand for themselves. Some characters have a special function:
| Character | Meaning |
| --- | --- |
| `.` | any one character |
| `^` | start of a line |
| `$` | end of a line |
| `*` | repeat the previous expression zero or more times |
| `[ β¦ ]` | one of the contained characters |
| `[^ β¦ ]` | a character that is not contained |
| `\` | change the meaning of the following character |
The pattern `error` matches anywhere in a line where that string occurs:
```bash
grep 'error' application.log
```
If the full line should consist only of `error`, start and end are anchored:
```bash
grep '^error$' application.log
```
Only lines that start with `error`:
```bash
grep '^error' application.log
```
Only lines that end with `error`:
```bash
grep 'error$' application.log
```
A dot is a metacharacter without escaping. The pattern `web.01` therefore matches not only `web.01`, but also `web-01` or `webX01`, for example.
For a literal dot the character must be escaped:
```bash
grep 'web\.01' hosts.txt
```
With `grep -E` the following pattern matches `web01.example.net`, for example, but not `db01.example.net` or `web1.example.net`.
```bash
grep -E '^web[[:digit:]]{2}\.example\.net$' hosts.txt
```
### Character classes and bracket expressions
A bracket expression describes a single character from a given set:
```bash
grep -E 'gr[ae]y' file.txt
```
The pattern matches `gray` and `grey`.
A range can be given with a hyphen:
```bash
grep -E '[0-9]' file.txt
```
For portable patterns POSIX character classes are preferable. They respect the active locale and make the intended character kind clearer:
| Character class | Meaning |
| --- | --- |
| `[[:digit:]]` | digit |
| `[[:alpha:]]` | alphabetic character |
| `[[:alnum:]]` | letter or digit |
| `[[:lower:]]` | lowercase letter |
| `[[:upper:]]` | uppercase letter |
| `[[:space:]]` | whitespace including tab and vertical space |
| `[[:blank:]]` | space or horizontal tab |
| `[[:xdigit:]]` | hexadecimal digit |
| `[[:punct:]]` | punctuation and special characters |
The double square brackets belong to the syntax:
```bash
grep -E '[[:digit:]]{4}' file.txt
```
A negated character class starts with `^` inside the brackets:
```bash
grep -E '[^[:digit:]]' file.txt
```
The pattern matches a character that is not a digit.
Evaluation of character classes and ranges can depend on the configured locale. If a script should work in a byte-oriented way with ASCII-like sorting, the locale can be set for the individual invocation:
```bash
LC_ALL=C grep -E '^[A-Z]+$' file.txt
```
β **Typical failure mode:** `[0-9]` and `[[:digit:]]` look similar, but they do not necessarily express the same character set under every locale. Use POSIX character classes when you want to describe the kind of character, not only a visible range.
### Repetitions and quantifiers
A quantifier always refers to the immediately preceding expression. That can be a single character, a character class or a grouped substructure.
In ERE the following quantifiers are available:
| Quantifier | Meaning |
| --- | --- |
| `*` | zero or more times |
| `+` | one or more times |
| `?` | zero or one time |
| `{n}` | exactly `n` repetitions |
| `{n,}` | at least `n` repetitions |
| `{n,m}` | between `n` and `m` repetitions |
One or more digits:
```bash
grep -E '[[:digit:]]+' file.txt
```
Exactly four digits:
```bash
grep -E '[[:digit:]]{4}' file.txt
```
Between two and four digits:
```bash
grep -E '[[:digit:]]{2,4}' file.txt
```
An optional suffix:
```bash
grep -E '^server(-backup)?$' systems.txt
```
The pattern matches `server` and `server-backup`.
Grouping decides what a repetition applies to:
```bash
grep -E '^(ab){3}$' file.txt
```
This pattern matches `ababab`. Without parentheses only the immediately preceding character would be repeated.
Special care is needed with apparently readable words:
```bash
grep -E 'error{2,}' file.txt
```
The quantifier belongs only to the last `r`. The pattern therefore requires `erro` followed by at least two further `r` characters. It does not mean that the word `error` must occur more than once.
Several complete occurrences are grouped:
```bash
grep -E '(error){2,}' file.txt
```
### Grouping and alternatives
Round parentheses gather several parts of an ERE expression into one unit. The vertical bar describes alternatives:
```bash
grep -E '^(INFO|WARNING|ERROR):' application.log
```
The expression requires exactly one of the three groups at the start of the line, followed by a colon. Alternatives should be grouped as tightly as possible. The following expression:
```bash
grep -E '^ERROR|WARNING$' application.log
```
means:
```markdown
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
β GREP: EXIT CODES AND STATUS HANDLING β
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ€
β β
β βββββββββββββββββββββ β
β β run grep β β
β βββββββββββ¬ββββββββββ β
β β β
β ββββββββββββββββββββΌβββββββββββββββββββ β
β βΌ βΌ βΌ β
β βββββββββββββ βββββββββββββ βββββββββββββ β
β β Match β β No β β Error β β
β β found β β match β β(file gone)β β
β βββββββ¬ββββββ βββββββ¬ββββββ βββββββ¬ββββββ β
β β β β β
β βΌ βΌ βΌ β
β Exit code 0 Exit code 1 Exit code 2 β
β β
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
```
For a complete line with exactly one of the two values, both alternatives must be anchored together:
```bash
grep -E '^(ERROR|WARNING)$' application.log
```
```markdown
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
β REGULAR EXPRESSION: PATTERN BREAKDOWN β
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ€
β β
β Pattern: ^web[[:digit:]]{2}\.example\.net$ β
β β
β ^ Start of line β
β web Literal string β
β [[:digit:]]{2} Exactly two digits (00-99) β
β \. Literal dot (escaped) β
β example Literal string β
β \. Literal dot (escaped) β
β net Literal string β
β $ End of line β
β β
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
```
Groupings also describe recurring structures. A syntactic pattern for an IPv4-like string is:
```bash
grep -E '([[:digit:]]{1,3}\.){3}[[:digit:]]{1,3}' access.log
```
The pattern recognises four digit groups separated by dots. It does not check whether each group is a valid value between 0 and 255.
β οΈ **Warning:** A regular expression that recognises the *form* of an IP address, a date or an email address does not automatically validate its operational validity. Form and meaning are two different checks.
### Extract matching text
With `grep -oE` you can print the part of a line that matches the full regular expression.
From a log line such as this:
```bash
2026-08-17T14:05:22Z login user=sebastian result=failed
```
the user value including the key can be extracted:
```bash
grep -oE 'user=[[:alnum:]_.-]+' application.log
```
Output:
```bash
user=sebastian
```
If only the value after `user=` should be processed further, a capture group in `grep` is not enough. `grep -o` prints the full match and not a freely chosen subgroup. The key can then be removed with a further tool:
```bash
grep -oE 'user=[[:alnum:]_.-]+' application.log | cut -d= -f2
```
Alternatively `sed` takes selection and replacement in one step:
```bash
sed -nE 's/.*user=([[:alnum:]_.-]+).*/\1/p' application.log
```
β **Typical failure mode:** Round parentheses do not make `grep` print only the contents of the group. They structure the pattern. `grep -o` still prints the entire matching expression.
### Select text with `sed`
`sed` processes an input stream line by line. Regular expressions can act as an address that decides which lines a command is applied to.
By default `sed` prints every processed line. With `-n` that automatic output is suppressed. The `p` command then prints only deliberately selected lines:
```bash
sed -n '/ERROR/p' application.log
```
With extended regular expressions:
```bash
sed -nE '/^(ERROR|WARNING):/p' application.log
```
For pure selection `grep` is usually shorter. `sed` becomes interesting when found text should be changed or reshaped at the same time.
### Substitutions with `sed`
The substitution command has this basic form:
```bash
s/pattern/replacement/flags
```
A simple replacement:
```bash
sed 's/localhost/db01.example.net/' application.conf
```
Without a further flag `sed` replaces only the first match per line. With `g` every occurrence inside the respective line is replaced:
```bash
sed 's/localhost/db01.example.net/g' application.conf
```
The command writes the result to `stdout`. The input file stays unchanged.
If the pattern contains many slashes, a different delimiter can be used:
```bash
sed 's#/var/www/html#/srv/www#g' application.conf
```
That is more readable than a sequence of escaped slashes.
In the replacement, `&` stands for the entire matched text:
```bash
sed -E 's/ERROR/[&]/g' application.log
```
`ERROR` therefore becomes `[ERROR]`.
Groups can be referenced with `\1` to `\9`. The following example replaces a password value but keeps the key:
```bash
sed -E 's/^(password=).*/\1[REDACTED]/' application.conf
```
The first group contains `password=`. `\1` inserts that part into the replacement.
```markdown
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
β REGULAR EXPRESSIONS: OR ALTERNATION β
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ€
β β
β Logical condition: β
β Line starts with ERROR OR line ends with WARNING β
β β
β ERE pattern with grep -E: β
β ^ERROR|WARNING$ β
β β
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
```
### Changing files with `sed`
The `-i` option writes changes directly into the given file. That should happen only after the same expression has been checked without `-i`.
First display only the planned output:
```bash
sed -E 's/^(timeout=)[[:digit:]]+/\130/' application.conf
```
Under GNU `sed`, `-i` can also create a backup copy:
```bash
sed -E -i.bak 's/^(timeout=)[[:digit:]]+/\130/' application.conf
```
The original file remains as `application.conf.bak`.
β οΈ **Warning:** `sed -i` replaces the file and can affect properties of symbolic links or implementation-dependent metadata. Check output and backup copy before you apply the command to several configuration files.
A backup copy is not yet verification. Compare both files afterwards:
```bash
diff -u application.conf.bak application.conf
```
Only the comparison shows whether exclusively the intended lines were changed.
### Quoting and escaping in the shell
A regular expression is not passed to `grep` or `sed` directly. First the shell processes the command line. Then the respective program interprets the remaining expression.
**Single quotes are therefore usually the safest choice for fixed regex expressions:**
```bash
grep -E '^[[:digit:]]{4}-[[:digit:]]{2}-[[:digit:]]{2}$' data.txt
```
Inside single quotes the shell changes neither `$` nor backslashes or asterisks.
**Double quotes, on the other hand, allow variable expansion:**
```bash
pattern='ERROR|WARNING'
grep -E "^($pattern):" application.log
```
That can be intended, but it joins shell and regex syntax. The content of the variable becomes part of the regular expression and must therefore be trustworthy and syntactically suitable.
The most important layers are:
```markdown
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
β ERE PATTERN: STRUCTURE AND ALTERNATIVES β
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ€
β β
β βββββββββββββββββββββ β
β β ^ β Start of line β
β βββββββββββ¬ββββββββββ β
β β β
β βΌ β
β βββββββββββββββββββββ β
β β (ERROR|WARNING) β One of the two β
β β β alternatives β
β βββββββββββ¬ββββββββββ β
β β β
β βΌ β
β βββββββββββββββββββββ β
β β $ β End of line β
β βββββββββββββββββββββ β
β β
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
```
π‘ **Tip:** If an expression does not work, check the layers separately. First print the pattern that is actually passed, then test only the regex, and add the `sed` replacement last. Three small errors stacked on top of each other rarely make an interesting puzzle. Usually they only cost an afternoon.
### Portability and GNU extensions
GNU `grep` and GNU `sed` support extra shorthand such as `\b`, `\w` or `\s`. These are convenient, but they do not always belong to portable POSIX ERE syntax.
For shell scripts that must work across distributions, POSIX character classes are usually the more reliable choice:
| GNU-like shorthand | More portable spelling |
| --- | --- |
| `\d` | `[[:digit:]]` |
| `\w` | `[[:alnum:]_]` |
| `\s` | `[[:space:]]` |
`grep -P` for Perl-compatible regular expressions is also not available on every system and is not part of the LPIC-1 core of this module.
`grep -E` and `sed -E` are the clear spellings for extended regular expressions. Whether an expression additionally uses GNU-specific extensions should be documented in scripts.
### Practical examples
Practice pattern: find ISO-like dates at the start of the line
```bash
grep -E '^[[:digit:]]{4}-[[:digit:]]{2}-[[:digit:]]{2}[[:space:]]' application.log
```
The expression only checks the form. A line with `2026-99-42` would match as well.
**Extract numeric HTTP status codes**
```bash
grep -oE 'status=[1-5][[:digit:]]{2}' access.log
```
**Recognise enabled or disabled values**
```bash
grep -E '^(enabled|disabled)=(yes|no)$' application.conf
```
**Normalise paths in an output**
```bash
sed -E 's#/var/www/[^[:space:]]+#/srv/www#g' report.txt
```
**Mask sensitive values in generated output**
```bash
sed -E 's/^(token=).*/\1[REDACTED]/' application.conf
```
The command does not change the input file as long as `-i` is missing.
β **Typical failure mode:** An ever larger regular expression is not automatically more precise. If validation, calculation or several interdependent rules are required, a regex may no longer be the right tool.
## Searching compressed files and archives
The previous module covered how files are compressed and bundled into archives with `tar`. This section builds directly on [archiving and compression](/en/lpic-1-serie/lpic-1-archiving-and-compression){.badge-link-text} and assumes the basics of `tar`, `gzip`, `bzip2` and `xz` explained there. Here the focus is targeted search of stored filenames and contents. For that search the distinction between compression and archiving is decisive.
A file such as `application.log.gz` contains a single compressed data stream. A file such as `logs.tar.gz` contains a `tar` archive compressed with `gzip`, in which several files and directories can sit.
These two formats need different search paths. `zgrep` is suitable for gzip-compressed single files. For a `tar.gz` archive, `tar` must first provide the desired archive member.
### Searching gzip-compressed files with `zgrep`
`zgrep` joins decompression of a gzip-compressed file with a subsequent search. The file is read during processing, but not unpacked permanently.
```bash
zgrep -nF "ERROR" application.log.gz
```
The options are passed through to `grep`. In this example `-n` adds the line number, while `-F` searches for the fixed string `ERROR`.
Several rotated log files can be searched together:
```bash
zgrep -HnF "connection refused" /var/log/my-application/*.gz
```
The shell expands `*.gz` to the existing files. `-H` makes the respective filename appear in the output.
Case can be ignored as usual:
```bash
zgrep -HinF "timeout" /var/log/my-application/*.gz
```
**Corresponding tools exist for other compression formats:**
| Format | Tool | Example |
| --- | --- | --- |
| gzip | `zgrep` | `zgrep -nF "ERROR" file.gz` |
| bzip2 | `bzgrep` | `bzgrep -nF "ERROR" file.bz2` |
| xz | `xzgrep` | `xzgrep -nF "ERROR" file.xz` |
Which options these wrappers support depends on the installed implementation. The respective man page shows which `grep` options are passed through.
π‘ **Tip:** Compressed logs do not have to be unpacked into a temporary file for a search. That saves write I/O and prevents a large uncompressed copy from sitting unnoticed next to the archive.
### Limits of direct search
A gzip file is decompressed as a stream. To inspect content at the end of the file, the preceding compressed stream must be processed. `zgrep` therefore cannot jump arbitrarily to a particular text line.
**The load comes mainly from:**
* reading the compressed file,
* decompression,
* evaluating the search pattern.
A large compressed file does not automatically cause correspondingly high RAM use. Processing normally happens as a stream. Runtime and CPU cost can still be substantial, especially with many archives or complex patterns.
With `gzip -t` you can check in advance whether a gzip stream can be read formally:
```bash
gzip -t application.log.gz
```
On success the command produces no normal output. Its exit code shows the result:
```bash
gzip -t application.log.gz
printf '%s\n' "$?"
```
An exit code of `0` stands for a successful check. Another value indicates an error.
β **Typical failure mode:** No output from `gzip -t` does not mean the contained log data is operationally complete or correct. The command checks the compressed stream, not the content of the logs.
### Searching archive members with `tar`
For a `tar` archive you can first list its table of contents:
```bash
tar -tf logs.tar
```
For a gzip-compressed archive:
```bash
tar -tzf logs.tar.gz
```
The output contains the names of the stored archive members. It does not contain their file contents. Particular entries can be filtered through a pipe:
```bash
tar -tzf logs.tar.gz | grep -E '\.log$'
```
This command searches for names that end in `.log`. It answers which log files are contained in the archive.
π‘ **Tip:** Display the archive listing first, fully or filtered. That gives you the exact internal path that `tar` expects for a subsequent content dump.
### Searching the content of one archive member
With `-O`, GNU `tar` writes the content of a selected archive member to `stdout` instead of extracting it as a file onto the filesystem.
Suppose the archive contains the member `var/log/my-application/error.log`:
```bash
tar -xOzf logs.tar.gz var/log/my-application/error.log
```
The output can be passed immediately to `grep`:
```bash
tar -xOzf logs.tar.gz var/log/my-application/error.log \
| grep -nF "ERROR"
```
**The flow consists of two clearly separated steps:**
1. `tar -xOzf` writes the selected archive member to `stdout`.
2. `grep` filters that stream for the pattern.
The internal path must be given as it is stored in the archive. A leading slash often does not belong to the member name, even if the original file lived under an absolute path.
**You therefore determine the correct name first with:**
```bash
tar -tzf logs.tar.gz
```
### Searching several archive members as a stream
GNU `tar` can write several matching members to `stdout`. With `--wildcards` you can use name patterns for that:
```bash
tar --wildcards -xOzf logs.tar.gz "*.log" \
| grep -nF "ERROR"
```
The contents of the matching files are then written one after another into the same stream. `grep` does not know the boundaries between archive members. Filenames and line numbers can therefore no longer be assigned reliably to the original member.
This form is therefore only suitable when the question is whether a pattern occurs *somewhere* in the selected contents. For a traceable assignment you should inspect archive members one by one.
β οΈ **Warning:** When several archive members are emitted together via `tar -xO`, a contiguous stream results. A matching line does not automatically contain the name of the original file.
### Why `zgrep` is not enough for `tar.gz`
**A common mistake is this command:**
```bash
zgrep -F "ERROR" logs.tar.gz
```
Here `zgrep` only removes the gzip compression. The result is not a normal text stream from a single log file, but a `tar` archive with headers, metadata and the file contents stored one after another.
Even if the command apparently returns a hit, a clean assignment to an archive member is missing. Binary data and tar metadata can also affect the evaluation.
The correct flow is:
```markdown
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
β SED: GROUPING AND BACKREFERENCE (\1) β
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ€
β β
β Input: password=secret β
β β
β Pattern: ^(password=).* β
β β β β
β β ββ Remaining content β
β βββββββββββββ Group 1 (\1) β
β β
β Replacement: \1[REDACTED] β
β Output: password=[REDACTED] β
β β
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
```
β **Typical failure mode:** The suffix `.tar.gz` names two processing steps: `tar` bundles several members, then `gzip` compresses the archive. `zgrep` only considers the outer gzip layer.
### Inspect the archive first
Before you read contents from an archive, you can check whether its listing can be processed without error:
```bash
tar -tzf logs.tar.gz >/dev/null
```
**Then you evaluate the exit code:**
```bash
status=$?
if [ "$status" -eq 0 ]; then
printf '%s\n' "Archive listing could be read"
else
printf '%s\n' "Archive could not be read completely" >&2
fi
```
This check shows whether `tar` could process the archive listing. It neither guarantees that all archived data is operationally complete, nor does it replace a restore test.
### Practical examples
Practice scenario: search for errors in a rotating gzip log file
```bash
zgrep -HnF "database connection failed" \
/var/log/my-application/*.gz
```
**List configuration files in an archive**
```bash
tar -tzf etc-backup.tar.gz | grep -E '\.conf$'
```
**Search a particular archived configuration**
```bash
tar -xOzf etc-backup.tar.gz etc/ssh/sshd_config \
| grep -nF "PermitRootLogin"
```
**Search for an error code in xz-compressed logs**
```bash
xzgrep -HnF "status=503" access.log.xz
```
**Check several archives for contained log files**
```bash
for archive in /backups/logs-*.tar.gz; do
printf '\n%s\n' "$archive"
tar -tzf "$archive" | grep -E '\.log$'
done
```
The loop shows separately for each archive which members end in `.log`. It does not yet search file contents.
π‘ **Tip:** When troubleshooting, separate the questions βWhich file is in the archive?β and βWhat content sits in that file?β. The first is answered by `tar -t`, the second by `tar -xO` together with `grep`.
## Combining `find`, `grep` and `xargs` safely
With `find` you search for files, while `grep` inspects their contents. As soon as both tools should work together, the output of one command must serve as input or an argument list for the next. You already know the basics of this data flow from [streams, pipes and redirections](/en/lpic-1-serie/lpic-1-streams-pipes-and-redirections){.badge-link-text}.
**A simple search chain consists of three steps:**
```markdown
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
β PROCESSING LAYERS: SHELL -> REGEX -> SED β
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ€
β β
β 1. Shell: β
β Protects or expands characters on the command line β
β β β
β βΌ β
β 2. Regex engine: β
β Interprets metacharacters such as ^ $ . * [ ] ( ) | β
β β β
β βΌ β
β 3. sed replacement: β
β Also interprets & and backreferences such as \1 β
β β
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
```
The pipe first transfers only a data stream. It does not decide whether a line is a filename, a search pattern or ordinary text. That meaning arises only through the receiving command.
### Searching files directly with `-exec`
If every found path should be passed immediately to `grep`, `find -exec` is the clearest solution:
```bash
find "$HOME/lpic-test" -type f -name "*.log" \
-exec grep -HnF -- "ERROR" {} +
```
`find` searches all files with the `.log` suffix. The found paths are passed to `grep`, which searches them for the fixed string `ERROR`.
The options used have the following meaning:
* `-H` shows the filename before each hit.
* `-n` adds the respective line number.
* `-F` treats `ERROR` as a fixed string and not as a regular expression.
* `--` ends processing of `grep` options.
* `{}` stands for the paths found by `find`.
* `+` gathers several paths into as few `grep` invocations as possible.
**A possible output looks like this:**
```bash
/home/user/lpic-test/system.log:14:ERROR ----> Service could not be started
/home/user/lpic-test/webserver.log:87:ERROR ----> Connection refused
```
π‘ **Tip:** If exactly one follow-up command should be run, `-exec ... {} +` is usually simpler than an extra pipe with `xargs`. The filenames are passed directly as arguments and do not have to be read again from a text stream.
The variant with `\;` already shown in the `find` section starts the command once for each hit:
```bash
find "$HOME/lpic-test" -type f -name "*.log" \
-exec grep -HnF -- "ERROR" {} \;
```
The result is comparable, but with many files far more processes are created. For `grep`, `stat`, `file` or `sha256sum` the batched form with `+` is therefore usually more sensible.
### Why `xargs` is needed
Not every command can read the filenames it should process from standard input. `grep`, for example, expects filenames as arguments after the search pattern:
```bash
grep -HnF -- "ERROR" system.log webserver.log
```
`xargs` reads entries from standard input and builds such an argument list from them. A simple combination could therefore look like this:
```bash
find "$HOME/lpic-test" -type f -name "*.log" |
xargs grep -HnF -- "ERROR"
```
For ordinary filenames this invocation works. It is still not reliable. By default `xargs` splits its input on spaces and newlines. A path like this is therefore broken incorrectly:
```bash
/home/user/lpic-test/old logs/system.log
```
**One filename becomes two arguments:**
```bash
/home/user/lpic-test/old
logs/system.log
```
`grep` then tries to open two files that do not exist.
β **Typical failure mode:** `find ... | xargs ...` is unsuitable for arbitrary filenames. Spaces, quotes, backslashes or newlines can change the split. That the command works with your own test files only proves that their names were polite just then.
### Passing filenames NUL-terminated
A Linux filename may contain almost every character. The null byte is the decisive exception and is therefore a unique separator.
`find` produces NUL-terminated output with `-print0`. `xargs` reads this format with `-0`:
```bash
find "$HOME/lpic-test" -type f -name "*.log" -print0 |
xargs -0 -r grep -HnF -- "ERROR"
```
The data flow then looks like this:
```markdown
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
β COMPRESSED LOGS: SINGLE FILE VS. ARCHIVE β
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ€
β β
β Compressed single file: β
β app.log ββgzipβββΊ app.log.gz ββzgrepβββΊ text β
β β
β Compressed archive (.tar.gz): β
β files ββtarβββΊ logs.tar ββgzipβββΊ logs.tar.gz β
β β β
β ββ tar -tzf (read archive listing) β β
β ββ tar -xOzf file | grep (stream content) β
β β
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
```
Spaces and newlines remain part of the respective filename. They are no longer interpreted as separators.
The `-r` option prevents GNU `xargs` from starting `grep` once without filenames if `find` returns no hit. It is a GNU extension and therefore not available on every Unix system. In the usual GNU/Linux environment it is still useful.
### Chaining several search steps
With `-l`, `grep` can print only the names of files that contain at least one hit. With GNU `grep -Z` each printed filename is terminated by a null byte. Safe hand-off is then preserved even in a longer command chain:
```bash
find "$HOME/lpic-test" -type f -name "*.log" -print0 |
xargs -0 -r grep -lZF -- "ERROR" |
xargs -0 -r stat --
```
Processing happens in clearly separated stages:
1. `find` supplies all matching log files.
2. `grep -lF` filters the files that contain the string `ERROR`.
3. `-Z` still prints the hits NUL-terminated.
4. `stat` shows information about the remaining files.
A pipeline is only as reliable as its weakest hand-off. If `grep` printed the filenames here again with ordinary newlines, the NUL termination would be broken.
### Controlling count and parallelism of invocations
`xargs` automatically gathers input into argument lists that stay within system limits. With `-n` you can additionally set how many inputs are passed to a single invocation at most:
```bash
find "$HOME/lpic-test" -type f -name "*.log" -print0 |
xargs -0 -r -n 5 stat --
```
`stat` then processes at most five files per invocation.
The `-P` option allows several parallel processes:
```bash
find "$HOME/lpic-test" -type f -name "*.iso" -print0 |
xargs -0 -r -n 1 -P 4 sha256sum --
```
This command computes up to four checksums at the same time. With many large files that can speed processing. The output then does not necessarily appear in the same order as the input.
Parallelism is only useful when the individual invocations work independently of each other. Several processes that change the same file or write the same output file do not produce a speed-up, but a lottery with process IDs.
π‘ **Tip:** Start without `-P` and first check that the command chain works correctly. Parallelisation should speed up a working process, not multiply its errors at the same time.
### Review search results before changes
The combination of `find`, `xargs` and a writing command can change many files in a short time. A wrongly set search criterion then also applies to many files β unfortunately with remarkable efficiency.
Therefore first check only the selection:
```bash
find "$HOME/lpic-test" -type f -name "*.tmp" -print
```
If file information should be inspected as well, the invocation still has no mutating effect:
```bash
find "$HOME/lpic-test" -type f -name "*.tmp" \
-exec stat -- {} +
```
Only when paths, file type and search scope are correct should the actual target command be added.
β οΈ **Warning:** Never join unchecked search results directly with deleting or overwriting commands. First check the same `find` query with `-print`, `stat` or a comparable read-only command. Search runs with elevated rights or in directories outside a test environment are especially critical.
**For most tasks a simple decision is therefore enough:**
* Use `find -exec ... {} +` when the hits go directly to a single command.
* Use `find -print0 | xargs -0` when you want to control argument building with options such as `-n`, `-P` or further processing steps.
* Do not use line-based hand-off as soon as arbitrary filenames can be processed.
## Summary exercises
The following exercises join the tools from this module into complete search workflows. You work exclusively in a dedicated exercise directory and need no root rights.
Some tasks additionally draw on [streams, pipes and redirections](/en/lpic-1-serie/lpic-1-streams-pipes-and-redirections){.badge-link-text} and on [archiving and compression](/en/lpic-1-serie/lpic-1-archiving-and-compression){.badge-link-text} from the previous module.
**The basic flow stays the same for all exercises:**
```markdown
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
β TAR ARCHIVE: INSPECT STRUCTURE AND FILTER β
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ€
β β
β logs.tar.gz β
β β β
β β tar -tzf (archive table of contents) β
β βΌ β
β List of all archive members β
β β β
β β grep '\.log$' (filter by file suffix) β
β βΌ β
β Names of matching log files β
β β
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
```
### Prepare a lab
First create a separate directory with a few log and configuration files:
```bash
lab="$HOME/lpic-search-lab"
mkdir -p "$lab"/{logs,config,archive}
printf '%s\n' \
"INFO Application started" \
"ERROR Connection failed" \
"INFO Retry" \
> "$lab/logs/application.log"
printf '%s\n' \
"WARN Disk space is running low" \
"ERROR Service unreachable" \
"INFO Check finished" \
> "$lab/logs/system old.log"
printf '%s\n' \
"Listen 80" \
"ServerName example.test" \
> "$lab/config/web.conf"
printf '%s\n' \
"Port 22" \
"PermitRootLogin no" \
> "$lab/config/ssh.conf"
```
Then create a compressed archive of the log files and a single file compressed with `gzip`:
```bash
tar -czf "$lab/archive/logs.tar.gz" \
-C "$lab" logs
gzip -k "$lab/logs/application.log"
```
Check the prepared files:
```bash
find "$lab" -type f -printf '%P\n'
```
**The output should contain these files:**
```bash
logs/application.log
logs/application.log.gz
logs/system old.log
config/web.conf
config/ssh.conf
archive/logs.tar.gz
```
π‘ **Tip:** Use the variable `$lab` for the exercises instead of typing the full path repeatedly. It applies only in the current shell. If you open a new terminal, you must set it again.
### Exercise 1: search files with several criteria
**Task:** Search the exercise directory for all regular, uncompressed files whose name ends in `.log`.
First try to assemble the matching `find` command yourself.
**Possible solution:**
```bash
find "$lab" -type f -name "*.log"
```
The result consists of two files:
```bash
/home/user/lpic-search-lab/logs/application.log
/home/user/lpic-search-lab/logs/system old.log
```
Then extend the search so that only files inside the `logs` directory are considered and the search does not descend into further subdirectories:
```bash
find "$lab/logs" -maxdepth 1 -type f -name "*.log"
```
**Now search exclusively for compressed files:**
```bash
find "$lab" -type f \( -name "*.gz" -o -name "*.xz" -o -name "*.bz2" \)
```
Grouping is important here. Without `\(` and `\)` the combinations would be evaluated differently, so `-type f` would not apply reliably to all name conditions.
β **Typical failure mode:** If `*.log` is not quoted, the shell already tries to expand the pattern in the current directory. `find` then no longer necessarily receives the intended search expression.
### Exercise 2: filter contents with `grep`
**Task:** Search all uncompressed log files for the fixed string `ERROR`. Filename and line number should be printed.
**Possible solution:**
```bash
grep -HnF -- "ERROR" "$lab"/logs/*.log
```
**The output should contain two hits:**
```bash
/home/user/lpic-search-lab/logs/application.log:2:ERROR Connection failed
/home/user/lpic-search-lab/logs/system old.log:2:ERROR Service unreachable
```
Then search with an extended regular expression for lines that start with `WARN` or `ERROR`:
```bash
grep -HnE -- "^(WARN|ERROR)" "$lab"/logs/*.log
```
**The expression consists of three parts:**
* `^` anchors the search at the start of the line.
* `(WARN|ERROR)` describes two possible strings.
* `-E` enables extended regular expressions.
Then determine only the names of the files in which `ERROR` occurs:
```bash
grep -lF -- "ERROR" "$lab"/logs/*.log
```
With `-c` you can additionally count how many matching lines each file contains:
```bash
grep -HcF -- "ERROR" "$lab"/logs/*.log
```
Output like this means that exactly one matching line was found in each file:
```bash
/home/user/lpic-search-lab/logs/application.log:1
/home/user/lpic-search-lab/logs/system old.log:1
```
### Exercise 3: extract data with `sed`
**Task:** From the configuration files print only the first value of each line. From `Listen 80`, for example, `Listen` should remain.
**Possible solution:**
```bash
sed -nE 's/^([^[:space:]]+).*/\1/p' "$lab"/config/*.conf
```
The output is:
* Port
* PermitRootLogin
* Listen
* ServerName
**The expression works as follows:**
```markdown
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
β TAR ARCHIVE: STREAM CONTENT AND SEARCH IT β
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ€
β β
β βββββββββββββββββββββββ β
β β logs.tar.gz β Archive stays packed β
β ββββββββββββ¬βββββββββββ β
β β β
β β tar -xOzf archive.tar.gz entry.log β
β βΌ β
β βββββββββββββββββββββββ β
β β Unpacked stream β Plain text on stdout β
β ββββββββββββ¬βββββββββββ β
β β β
β β grep pattern β
β βΌ β
β βββββββββββββββββββββββ β
β β Filtered lines β β
β βββββββββββββββββββββββ β
β β
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
```
The `-n` option suppresses `sed`'s normal output. Only lines to which the replacement was applied appear through the trailing `p`.
Now filter exclusively lines that start with `ServerName` or `PermitRootLogin`:
```bash
sed -nE '/^(ServerName|PermitRootLogin)[[:space:]]/p' \
"$lab"/config/*.conf
```
This task does not change any files. `sed` writes the result only to standard output as long as neither `-i` nor a redirection into the source file is used.
β οΈ **Warning:** Do not use `sed -i` to experiment with important configuration files. The option changes the files directly. Check an expression first without `-i` and, with real configurations, work with a backup or version control.
### Exercise 4: search compressed files and archives
A single `.gz` file and a `.tar.gz` archive look similar by their suffix, but they contain different structures. Exactly that distinction was covered in [archiving and compression](/en/lpic-1-serie/lpic-1-archiving-and-compression){.badge-link-text}.
First search the single compressed log file with `zgrep`:
```bash
zgrep -HnF -- "ERROR" "$lab/logs/application.log.gz"
```
`zgrep` decompresses the stream internally and passes it to `grep`. The file stays unchanged.
Then show the archive contents:
```bash
tar -tzf "$lab/archive/logs.tar.gz"
```
The output contains the stored directory and file names:
```bash
logs/
logs/application.log
logs/system old.log
```
Now search inside the archived file `logs/application.log` without unpacking the archive permanently:
```bash
tar -xOzf "$lab/archive/logs.tar.gz" \
"logs/application.log" |
grep -nF -- "ERROR"
```
`tar -xOzf` writes the content of the selected archive file to standard output. The pipe forwards that stream immediately to `grep`.
```markdown
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
β DIFFERENCE: GZIP STREAM VS. TAR ARCHIVE β
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ€
β β
β Wrong: logs.tar.gz βββΊ zgrep βββΊ binary tar header β
β (zgrep does not understand tar containers!) β
β β
β Correct: logs.tar.gz βββΊ tar -xO βββΊ grep βββΊ matches β
β (tar extracts the member cleanly to stdout) β
β β
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
```
The output contains no filename, because `grep` here processes a stream and not a named file:
```bash
2:ERROR Connection failed
```
π‘ **Tip:** First check the exact name of the archive member with `tar -tzf`. `tar -xOzf` needs the path stored in the archive, not the original absolute file path.
### Exercise 5: join `find`, `grep` and `xargs`
This task picks up the data flow from [streams, pipes and redirections](/en/lpic-1-serie/lpic-1-streams-pipes-and-redirections){.badge-link-text} again. This time filenames are passed on, not only ordinary text lines.
First search all log files for `ERROR` with `find -exec`:
```bash
find "$lab/logs" -type f -name "*.log" \
-exec grep -HnF -- "ERROR" {} +
```
Then run the same search with a NUL-terminated hand-off to `xargs`:
```bash
find "$lab/logs" -type f -name "*.log" -print0 |
xargs -0 -r grep -HnF -- "ERROR"
```
Both commands must also process the file `system old.log` correctly. The space in the filename must not split it into several arguments.
For comparison, test the unsafe variant:
```bash
find "$lab/logs" -type f -name "*.log" |
xargs grep -HnF -- "ERROR"
```
For `system old.log`, `grep` should print error messages because `xargs` splits the path at the space. Exactly this controlled failure shows why `-print0` and `-0` belong together.
Finally check which log files actually contain `ERROR`, and pass only their names to `stat`:
```bash
find "$lab/logs" -type f -name "*.log" -print0 |
xargs -0 -r grep -lZF -- "ERROR" |
xargs -0 -r stat --
```
**NUL termination is preserved through the entire chain:**
```markdown
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
β PIPELINE COUPLING: FIND + XARGS / -EXEC + GREP β
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ€
β β
β βββββββββββββββββ ββfilenamesβββΊ βββββββββββββββββ β
β β find β β xargs / -exec β β
β β (file tree) β β (arguments) β β
β βββββββββββββββββ βββββββββ¬ββββββββ β
β β β
β βΌ β
β βββββββββββββββββ β
β β grep β β
β β (text content)β β
β βββββββββββββββββ β
β β
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
```
### Control questions
Answer the following questions without looking up the previous sections:
1. Why may `locate` not be current for files created immediately beforehand?
2. Which `find` option restricts the search to regular files?
3. When is `grep -F` the better choice compared with a regular expression?
4. What meaning do `^` and `$` have in a regular expression?
5. Why can `zgrep` search a `.gz` file, but not readily every file in a `.tar.gz` archive?
6. Why must `find -print0` and `xargs -0` be used together?
7. When is `find -exec ... {} +` simpler than a combination with `xargs`?
8. Why should a search query first be checked read-only before a writing or deleting action?
The answers do not come from memorising individual options, but from the respective data flow:
* Where do the filenames come from?
* How are they separated?
* Which command interprets them as arguments or as contents?
## Command Reference (Cheatsheet)
The following table summarises the main tools and options for file and text search:
| Command / syntax | Function and use | Typical example |
|---|---|---|
| `find [path] -name "pattern"` | Search files by name in the filesystem | `find /etc -name "*.conf"` |
| `find [path] -iname "pattern"` | Case-insensitive name search | `find /var/log -iname "*error*"` |
| `find [path] -type [f\|d\|l]` | Filter by file type (f=file, d=directory, l=link) | `find /tmp -type f -empty` |
| `find [path] -mtime [-n\|+n]` | Search by modification time (days; `-n` younger, `+n` older) | `find /var/log -type f -mtime -7` |
| `find [path] -size [+n\|-n][c\|k\|M\|G]` | Search by file size | `find /var -type f -size +100M` |
| `find [path] -perm [mode]` | Filter by permissions | `find /bin -perm -4000` |
| `find ... -exec [cmd] {} +` | Run a command efficiently batched with hits | `find /etc -name "*.conf" -exec grep -H "Port" {} +` |
| `find ... -print0 \| xargs -0` | Safe hand-off even with spaces in the path | `find . -type f -print0 \| xargs -0 grep -l "ERROR"` |
| `locate [term]` | Fast search over the indexed database state | `locate nginx.conf` |
| `locate -e [term]` | Show only hits that still exist in the filesystem | `locate -e backup.tar.gz` |
| `updatedb` | Update the `locate`/`plocate` index database | `sudo updatedb` |
| `grep [options] "pattern"` | Search regular expressions line by line in text | `grep -E "^(error\|warn)" /var/log/syslog` |
| `grep -F "string"` | Search a fixed string (faster, no regex) | `grep -F "[DEBUG]" app.log` |
| `grep -i -n -v` | `-i` ignores case, `-n` line numbers, `-v` inverts | `grep -v "^#" /etc/ssh/sshd_config` |
| `grep -r -l "pattern" [dir]` | Recursive search, print only matching filenames | `grep -rl "PermitRootLogin" /etc/ssh/` |
| `zgrep` / `bzgrep` / `xzgrep` | Search compressed single files directly | `zgrep -i "failed" /var/log/auth.log.1.gz` |
| `tar -xOzf [archive] [path] \| grep` | Stream an archive member to stdout and filter with grep | `tar -xOzf logs.tar.gz logs/app.log \| grep "ERR"` |
| `sed -n 's/pattern/replacement/p'` | Extract and modify filtered text lines | `sed -n 's/^user=\([^ ]*\).*/\1/p' auth.log` |
## Further Resources
The following reference table leads to official documentation, exam objectives and manuals:
| Resource | Description | Type / link |
|---|---|---|
| LPI 101-500 exam objectives | Official exam objectives for LPIC-1 topics 103.3 and 103.7 | [LPI Exam Objectives](https://www.lpi.org/our-certifications/exam-101-objectives/){.badge-link-text} |
| LPI learning materials | Official learning materials on file management and regex | [LPI Learning Portal](https://learning.lpi.org/en/learning-materials/101-500/){.badge-link-text} |
| GNU Findutils manual | Reference for `find`, `xargs`, `locate` and `updatedb` | [GNU Findutils Manual](https://www.gnu.org/software/findutils/manual/){.badge-link-text} |
| GNU Grep manual | Full documentation of BRE, ERE and grep flags | [GNU Grep Manual](https://www.gnu.org/software/grep/manual/){.badge-link-text} |
| GNU Sed manual | Detailed command and addressing reference for the stream editor `sed` | [GNU Sed Manual](https://www.gnu.org/software/sed/manual/){.badge-link-text} |
| POSIX regex specification | Standardised POSIX character classes (`[:digit:]`, `[:space:]` and others) | [POSIX Regular Expressions](https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap09.html){.badge-link-text} |
| plocate man page | Documentation of the deterministic, very fast plocate indexer | [plocate Manual](https://plocate.sesse.net/){.badge-link-text} |
| LPIC-1: navigation and filesystem | Module 2 of the series: basic navigation and filesystem commands | [LPIC-1 navigation and filesystem](/en/lpic-1-serie/lpic-1-basic-navigation-and-filesystem-commands){.badge-link-text} |
| LPIC-1: streams and pipes | Module 5 of the series: streams, pipes and redirections | [LPIC-1 streams and pipes](/en/lpic-1-serie/lpic-1-streams-pipes-and-redirections){.badge-link-text} |
| LPIC-1: archiving and compression | Module 6 of the series: archives with tar, gzip, bzip2 and xz | [LPIC-1 archiving and compression](/en/lpic-1-serie/lpic-1-archiving-and-compression){.badge-link-text} |
## Conclusion
Part 7 of the LPIC-1 series covered targeted search for files and contents on Linux, narrowing results and passing the found data on safely.
With `find` you can search the current filesystem state by name, type, size, timestamps and further properties. `locate` and `plocate` return their results from a database instead. That is faster, but it requires a current file index.
Content search centres on `grep`. Fixed strings can be handled unambiguously with `-F`, while regular expressions enable more complex patterns. With `sed` you can then not only display matching data, but extract or reshape it deliberately.
Compressed data remains searchable as well. `zgrep`, `bzgrep` and `xzgrep` work with individual compressed files. For a `tar` archive you must first determine the contained files and then pass the desired content via standard output. This module builds directly on the previously covered [archiving and compression](/en/lpic-1-serie/lpic-1-archiving-and-compression){.badge-link-text}.
### The key points at a glance
`find` searches the filesystem, `locate` searches an index and `grep` searches inside files. Regular expressions do not describe the search location, but the pattern that should be recognised in text. For compressed files the data structure decides whether a tool such as `zgrep` is enough or an archive must first be opened with `tar`.
Combined with [streams, pipes and redirections](/en/lpic-1-serie/lpic-1-streams-pipes-and-redirections){.badge-link-text}, these tools form complete search chains. `find -exec ... {} +` is suitable for passing hits directly to a command. If you need `xargs`, the combination of `-print0` and `-0` protects filenames with spaces or newlines from a faulty split.
π‘ **Tip:** First limit the search scope, then filter the contents. A tight search not only yields clearer results, it also avoids unnecessary access to directories and files that play no role for the actual task.
The tools covered belong to the basic toolkit both for the [LPIC-1 exam](https://www.lpi.org/our-certifications/lpic-1-overview){.badge-link-text} and for practical Linux administration. The point is not to know every option by heart. You should be able to recognise which data source is searched, how current it is and in which form the result is passed to the next command.
All parts published so far are in the [LPIC-1 series overview](/en/category/lpic-1-serie){.badge-link-text}.
π **To the course overview:** [All LPIC-1 articles and modules](/en/category/lpic-1-serie){.badge-link-text}