Physical Address
Metro Manila, Philippines
Physical Address
Metro Manila, Philippines
Linux file permissions are usually the first thing administrators check when they encounter a file or directory they cannot modify.
We run:
ls -l
and look for familiar permissions such as:
rwxr-xr-x
Sometimes that is not enough.
During our troubleshooting of a compromised Zimbra mail server, we investigated /tmp because some filesystem behavior did not make sense. The normal permissions appeared writable, but another command revealed something much more interesting:
lsattr -d /tmp
The result was:
----ia-------e-- /tmp
The important characters were:
i
a
e
The i attribute means immutable.
The a attribute means append-only.
The e attribute indicates that extents are being used to map the object’s blocks on an ext4 filesystem.
Finding both i and a on /tmp was unusual and important because /tmp is expected to be available to programs that need temporary files. The Filesystem Hierarchy Standard specifically defines /tmp as a location made available to programs for temporary files.
This article explains what those attributes mean, how they differ from normal Linux permissions, how lsattr and chattr work, and why you should never remove filesystem attributes blindly from an important system directory.
During our Zimbra troubleshooting, we checked /tmp using several commands.
The standard permission information showed that /tmp had the expected general writable permissions and sticky bit.
We also checked its filesystem attributes:
lsattr -d /tmp
The output was:
----ia-------e-- /tmp
The -d option is important. It tells lsattr to display the attributes of the directory itself instead of listing attributes for files contained inside it.
We also confirmed that the root filesystem was ext4.
The unusual part was therefore not the e attribute.
The unusual part was that /tmp itself had both:
i
a
set.
This article does not claim that we established exactly who set those attributes or whether they were directly caused by the compromise. They were an abnormal filesystem condition discovered while troubleshooting the affected server.
Linux administrators are familiar with traditional permission bits.
For example:
ls -ld /tmp
might display:
drwxrwxrwt 10 root root 4096 Sep 13 10:30 /tmp
The important portion is:
drwxrwxrwt
These permissions control normal read, write, and execute access for:
owner
group
others
The final t is the sticky bit.
On /tmp, the sticky bit allows multiple users to create files while generally preventing users from deleting files owned by somebody else.
Linux file attributes operate at another layer.
They can change how the filesystem treats a file or directory even when normal Unix permissions appear to allow an operation.
This explains an important troubleshooting situation:
Permissions say: writable
Filesystem attributes say: immutable
Result: writing or modifying can still fail
The Linux kernel exposes these as inode flags. lsattr reads them and chattr modifies supported flags.
You will sometimes hear lsattr attributes described informally as “extended attributes.”
Technically, that can cause confusion.
The flags manipulated by:
lsattr
chattr
are Linux file attributes or inode flags.
Linux also has a separate feature called extended attributes, commonly shortened to xattrs.
Those can contain named values such as:
user.example
security.selinux
system.posix_acl_access
and are commonly inspected with tools such as:
getfattr
setfattr
The systemd documentation explicitly notes that Linux file attributes such as those manipulated by chattr are a different concept from extended attributes.
So throughout this article, when we discuss:
i
a
e
we are talking about filesystem inode flags displayed by lsattr.
lsattrThe command:
lsattr
lists supported file attributes.
For example:
lsattr /etc/example.conf
might return:
--------------e------- /etc/example.conf
For a directory itself, use:
lsattr -d /tmp
Without -d, lsattr normally examines the directory’s contents instead.
The Linux lsattr manual documents -d specifically for treating directories like normal files rather than listing their contents.
You can also use:
lsattr -a /tmp
to include hidden entries.
For recursive inspection:
lsattr -R /some/path
Be careful with recursive commands on large production filesystems.
i Attribute MeansThe most important attribute in our /tmp result was:
i
The i flag means immutable.
For a regular file, immutable means the file cannot normally be modified, deleted, renamed, linked to, or opened for writing.
The Linux chattr documentation states that most metadata cannot be modified either. Only a privileged process with CAP_LINUX_IMMUTABLE can set or clear this attribute.
For example:
touch /tmp/example
chattr +i /tmp/example
After that:
echo "test" >> /tmp/example
should fail.
So should:
rm /tmp/example
even if the ordinary permission bits would otherwise allow the operation.
The immutable flag therefore provides restrictions beyond:
rwx
permissions.
Our incident involved:
/tmp
itself, not simply one file inside /tmp.
A directory is essentially a structure containing directory entries that map names to filesystem objects. Changing those entries requires modifying the directory. The Linux kernel’s ext4 documentation describes directories as structures containing these directory entries.
If the directory itself is immutable, normal attempts to alter its directory contents can fail.
That can affect operations such as:
creating files
removing files
renaming entries
creating subdirectories
removing subdirectories
This makes an immutable /tmp particularly problematic.
A large number of programs expect to be able to create temporary files there.
a Attribute MeansThe second important flag was:
a
This means append-only.
For a regular file, append-only means data can be added to the end of the file, but existing contents cannot normally be overwritten.
For example:
chattr +a example.log
allows:
echo "new entry" >> example.log
but prevents ordinary operations that would overwrite or truncate the existing data.
The Linux manual states that only a privileged process with CAP_LINUX_IMMUTABLE can set or clear the append-only flag.
The behavior becomes especially interesting when a is applied to a directory.
Linux documentation states that the append-only flag on a directory prevents files from being removed from that directory through operations such as unlink() and rename().
That means something like:
/tmp
being append-only can interfere with normal temporary-file cleanup.
Programs might create temporary files but later be unable to remove or rename them.
System cleanup tools can also be affected.
This is important because systemd-tmpfiles normally performs creation, removal, and cleanup operations involving temporary and volatile locations including /tmp and /var/tmp.
i and a Are Both Set?Our actual output was:
----ia-------e-- /tmp
So /tmp had both:
i = immutable
a = append-only
The immutable flag is the stronger restriction.
If the directory cannot be modified because it is immutable, the append-only behavior does not suddenly make normal modification possible.
The combination is therefore highly restrictive.
For a directory whose purpose is providing temporary storage to applications, that condition deserves immediate investigation.
e Attribute MeansOur output also contained:
e
This was:
----ia-------e--
The e flag means the object uses extents for mapping filesystem blocks.
The Linux chattr documentation states that this flag indicates extent-based block mapping and that it cannot be removed using chattr.
The ext4 documentation also lists:
e = extents format
as an ext4 filesystem attribute.
Therefore, on an ext4 filesystem:
e
by itself is not an indication of compromise.
This is an important distinction.
When reading:
----ia-------e--
we should not treat every letter as suspicious.
In our troubleshooting case:
i required investigation
a required investigation
e expected ext4-related filesystem information
lsattr AttributesLinux supports several other file attributes, although availability depends on the filesystem.
Some useful ones administrators may encounter include:
| Attribute | Meaning |
|---|---|
i | Immutable |
a | Append-only |
A | Do not update access time |
d | Exclude from dump backups |
D | Synchronous directory updates |
e | Extent format |
j | Data journaling on ext3/ext4 |
S | Synchronous updates |
I | Indexed directory |
V | fs-verity enabled |
Not every filesystem supports every flag.
The chattr documentation explicitly warns that filesystem support varies.
You should therefore identify the filesystem before interpreting attributes.
For example:
findmnt -T /tmp
or:
df -T /tmp
can help determine what filesystem contains /tmp.
chattr Changes Attributeschattr changes supported Linux file attributes.
The basic syntax is:
chattr OPERATOR ATTRIBUTE PATH
The three primary operators are:
+ add an attribute
- remove an attribute
= replace the selected attributes
For example:
chattr +i important.conf
adds the immutable flag.
This:
chattr -i important.conf
removes it.
And:
chattr +a application.log
adds append-only behavior.
The chattr manual documents these +, -, and = operations.
chattr -ia /tmpAfter seeing:
----ia-------e-- /tmp
the obvious reaction might be:
chattr -ia /tmp
That would remove both:
i
a
from /tmp.
But that should not be your first step.
Before changing filesystem attributes on an important production directory, determine why they are there.
Record the existing state first:
lsattr -d /tmp
Then record permissions:
ls -ld /tmp
Check ACLs:
getfacl -p /tmp
Check metadata:
stat /tmp
Identify the filesystem:
findmnt -T /tmp
Check the mount state:
mount | grep ' on / '
or:
findmnt /
If the system is part of an active security incident, preserve these results before modifying anything.
This distinction is important.
Suppose a command fails with:
Operation not permitted
You might assume the filesystem has become read-only.
Check:
findmnt /
or:
mount | grep ' on / '
A filesystem mounted as:
rw
can still contain an immutable file or directory.
For example:
Filesystem: read-write
/tmp permissions: rwxrwxrwt
/tmp attribute: immutable
can still result in operations failing against /tmp.
Normal permissions, filesystem mount state, ACLs, and inode flags are separate controls.
You need to check each one.
/tmp Is Important to ApplicationsThe Filesystem Hierarchy Standard defines /tmp as the directory for temporary files and states that it must be available to programs that require temporary files.
Applications may use temporary storage for activities such as:
extracting temporary data
building intermediate files
creating sockets or lock files
generating temporary configuration
performing software installation
processing uploads
sorting or transforming data
This means abnormal restrictions on /tmp can produce failures that seem unrelated to the filesystem.
You might see an application report:
Cannot create temporary file
or:
Operation not permitted
while /tmp still appears to have:
drwxrwxrwt
permissions.
That is exactly why lsattr is worth checking when normal permission troubleshooting does not explain the behavior.
Software installation and maintenance tools may use temporary files during package extraction, script execution, configuration generation, or other intermediate operations.
If /tmp cannot be modified normally, a package operation may fail even when:
df -h
shows available disk space and:
ls -ld /tmp
shows normal permissions.
Do not immediately conclude:
APT is broken.
or:
The package database is corrupt.
Check the underlying filesystem conditions too.
A useful troubleshooting sequence is:
ls -ld /tmp
lsattr -d /tmp
getfacl -p /tmp
df -h /tmp
df -T /tmp
findmnt -T /tmp
These commands examine different parts of the problem.
Databases and supporting tools can also use temporary storage.
The exact location depends on the database software and configuration.
If a database or maintenance process is configured to use /tmp, restrictive attributes on the directory may prevent it from:
creating temporary files
renaming temporary files
removing temporary files
creating sockets
performing maintenance operations
This can lead administrators down the wrong troubleshooting path.
A database service may fail, but the database engine itself may not be the root cause.
Always inspect the error logs before changing database configuration.
If the error refers to:
/tmp
temporary file
permission denied
operation not permitted
cannot create
cannot unlink
cannot rename
check both normal permissions and filesystem attributes.
Temporary files are normally expected to come and go.
On systemd-based Linux systems, systemd-tmpfiles can create, remove, and clean temporary or volatile files and directories. Its documentation specifically mentions locations such as:
/run
/tmp
/var/tmp
among the areas commonly managed by the mechanism.
Now consider an append-only directory.
Linux states that an append-only directory prevents file removal through operations such as unlink() and rename().
Cleanup operations can therefore fail.
An immutable directory is even more restrictive.
One of the confusing aspects of immutable files is that normal root privileges do not make the flag irrelevant.
The kernel documentation states that immutable restrictions apply even to the superuser during ordinary modification attempts. A privileged process with the required CAP_LINUX_IMMUTABLE capability can set or clear the relevant flag.
So this:
rm protected-file
may fail as root.
But root can normally remove the immutable flag first:
chattr -i protected-file
and then remove the file.
That difference is important.
The immutable flag is not saying:
root can never change this
It means normal file-modification operations are blocked until an appropriately privileged process removes the flag.
If you want to understand these attributes, do not experiment on /tmp or another system directory.
Create a lab directory:
mkdir ~/attr-test
cd ~/attr-test
Create a file:
echo "original" > example.txt
Inspect it:
lsattr example.txt
Set immutable:
sudo chattr +i example.txt
Check again:
lsattr example.txt
You should now see an i.
Try:
echo "changed" >> example.txt
or:
rm example.txt
The operation should fail.
Remove the attribute:
sudo chattr -i example.txt
Then verify:
lsattr example.txt
This is a much safer way to learn chattr than experimenting with:
/etc
/var
/tmp
/opt
on a production server.
chattr Blindlychattr supports:
-R
for recursive operations.
For example:
chattr -R -i /some/directory
can remove immutable attributes recursively.
That can be dangerous.
You may unintentionally change attributes on:
application data
security-sensitive files
database files
configuration files
backup repositories
system-managed files
The original attribute may have been intentionally set as a security or operational control.
Always inspect first.
For example:
lsattr -R /some/directory
and record the state before changing anything.
chattr = Unless You Understand the ConsequencesThis syntax:
chattr =i file
does not simply add immutable.
The = operator sets the selected file attributes as the resulting attribute set supported by that operation.
That is different from:
chattr +i file
which only adds i.
When troubleshooting production systems, targeted changes are safer than broad attribute replacement.
e Flag Should Usually Be Left AloneBecause our /tmp output contained:
e
an administrator might attempt:
chattr -e /tmp
That is not appropriate.
Linux documents e as indicating extent-based block mapping and specifically says it may not be removed using chattr.
On our ext4 filesystem, e was not the problematic part of:
----ia-------e--
The attributes requiring investigation were:
i
a
If a file or directory behaves as though it is not writable even though normal permissions look correct, check several layers before modifying anything.
Start with:
ls -ld /path
Then:
lsattr -d /path
Check ACLs:
getfacl -p /path
Inspect metadata:
stat /path
Identify the filesystem:
findmnt -T /path
Check capacity:
df -h /path
Check inode availability:
df -i /path
For /tmp, you can also perform a non-destructive creation test when appropriate:
touch /tmp/test-file-$$
and then:
rm /tmp/test-file-$$
Do not perform this test if you are preserving a compromised system for forensic analysis, because even a simple test modifies filesystem state.
The presence of:
i
or:
a
does not automatically mean malware.
An administrator might intentionally use immutable on a sensitive configuration file:
chattr +i important.conf
Likewise, append-only behavior can be useful when you want existing file contents protected from ordinary overwrite or truncation.
The security issue is context.
Ask:
Who set this?
Why was it set?
Is it documented?
Does the application expect it?
When was it changed?
Does it correspond with the incident timeline?
In our Zimbra case, finding i and a on /tmp was relevant because the server was already under security and filesystem investigation.
The same properties that make immutable attributes useful for protection can make them useful to an attacker.
For example, malicious files can be marked immutable to make ordinary deletion fail.
An administrator might try:
rm malicious-file
and receive:
Operation not permitted
even as root.
Before assuming filesystem corruption, inspect:
lsattr malicious-file
If it shows:
----i-----------
then the immutable flag may explain the behavior.
But do not immediately run:
chattr -i malicious-file
rm malicious-file
during an incident.
Preserve the file, its metadata, its hash, and relevant process information first.
The important command in our troubleshooting was:
lsattr -d /tmp
which returned:
----ia-------e-- /tmp
We could interpret that as:
| Attribute | Meaning | Assessment in Our Case |
|---|---|---|
i | Immutable | Abnormal for /tmp, required investigation |
a | Append-only | Abnormal for /tmp, required investigation |
e | ext4 extent format | Expected filesystem-related attribute |
This explained why simply looking at:
ls -ld /tmp
was not enough.
Normal Unix permissions and ACLs are only part of Linux access control.
The inode flags shown by lsattr can impose additional restrictions.
From our actual troubleshooting, we confirmed that:
/tmp was on the affected Linux server.
/tmp had normal Unix-style access permissions.
/tmp was on the ext4 root filesystem.
lsattr -d /tmp returned attributes containing i, a, and e.
The server was already being investigated for compromise and filesystem problems.
We did not establish from the lsattr output alone:
who set the i and a flags
exactly when they were set
whether malware set them
whether an administrator set them earlier
whether they directly caused every Zimbra or MySQL problem we observed
That distinction matters.
The correct conclusion was:
/tmphad abnormal filesystem attributes that could interfere with normal operations and required further investigation.
It would have been incorrect to state:
The malware definitely set
/tmpimmutable.
unless other evidence proved it.
The biggest lesson from this troubleshooting case was that Linux permissions do not tell the entire story.
A directory can appear writable through:
ls -ld
while filesystem attributes still prevent modifications.
The i attribute can make a file or directory immutable.
The a attribute can restrict it to append-oriented behavior and, on directories, prevent existing entries from being removed or renamed.
The e attribute on ext4 indicates extent-based storage and is not itself suspicious.
When Linux reports unexplained:
Operation not permitted
or applications suddenly fail to create, delete, or rename files, check:
lsattr
before changing permissions repeatedly.
And if you discover unusual attributes on an important path such as:
/tmp
/etc
/var
/opt
do not blindly remove them.
Document the state, understand why the flags exist, correlate the timestamps and incident evidence, and only then decide whether changing them is appropriate.
Sometimes the problem is not:
chmod
at all.
It is:
chattr
Linux lsattr(1) Manual
Documents lsattr, including the -d, -R, and -a options and its purpose for displaying ext2, ext3, and ext4 file attributes.
Linux lsattr(1) manual
Linux chattr(1) Manual
Documents immutable, append-only, extents, journaling, no-atime, synchronous updates, and other Linux file attributes.
Linux chattr(1) manual
Linux FS_IOC_GETFLAGS and FS_IOC_SETFLAGS Manual
Documents the underlying Linux inode flags, including the exact semantics of immutable and append-only attributes on files and directories.
Linux inode flag documentation
Linux ext4 Documentation
Documents ext4 file attributes, including append-only, immutable, journaling, and extent format.
Linux ext4(5) manual
Linux Kernel ext4 Inode Documentation
Documents ext4 inode flags, including EXT4_IMMUTABLE_FL, EXT4_APPEND_FL, and other filesystem-level flags.
Linux kernel ext4 inode documentation
systemd tmpfiles.d Documentation
Explains the management of temporary and volatile files and explicitly distinguishes Linux file attributes from extended attributes.
Linux tmpfiles.d documentation
Filesystem Hierarchy Standard, /tmp
Defines /tmp as a directory that must be available to programs requiring temporary files.
Filesystem Hierarchy Standard 3.0