AIX, like all other industry leading operating systems has a built-in feature of Auditing. This Auditing subsystem is part of base operating system and provides system administrators the features to record information pertinent to system security. This information is essential for the system administrators to prevent potential violations of the system security policy.
Any occurrence on the AIX servers relevant to system security is considered an auditable event. The set of auditable events on the system defines which occurrences can actually be audited and the granularity of the auditing provided.
Hence we can say that main concept of auditing is to detect any occurrence of auditable event record information pertaining to the set of auditable events, and process this information to examine audit trails and generate periodic reports.
An auditing subsystem should also provide feature to monitor the audit trail in real time for generating alerts to immediate security threats. AIX audit subsystem is no exception. It is capable of recording audit events for long term analysis by system administrators and security administrators as well as it can also provide real time auditing.
In this article, I will explore AIX auditing subsystem in detail, covering its different configuration aspects. I will also take my readers to basic configuration steps, so that they can configure this freely available feature of AIX 5L for auditing different system events happening on their AIX servers.
1.AIX Auditing Subsystem Components & Structure
As I described earlier that an audit is an examination of a group, individual account or activity and the auditing Subsystems provides a means of tracing and recording what is happing on your system.
This Subsystem also provides the means of security related information and to alert System and security administrators about potential and actual violations of the system security policy. For example a system administrator would like to be identified either immediately or at day end that some internal or external intruder has tried to change some critical files on your system ( like database log files or system critical log files like smit.log file or root user shell history file etc). Here comes the AIX auditing to help administrators.
Let’s start with components description for AIX auditing. Main component is auditing configuration file ( /etc/security/audit/config ).Whenever you start the auditing daemon on AIX , this configuration file is read. This file contains information, such as
• Mode
Mode represents the data collection method, as used by auditing daemon .There are two different modes of AIX auditing , which are discussed in detail in next section of this article. These modes are
– Bin Mode
– Stream Mode
• Events
Events are systems defined activity. Here are two examples
– USER_SU give you information about whether a user tries to su another user; this event is associated with General class, by default.
– CRON_Start event gives you information whether a cron job has started or not.
• Customized Events( if any):
The Systems Administrator can also define his own Customized Events relating to system kernel activity and also to the critical users activates. For example, in this article I have defined some customized events related to smit activities as performed by root user.
I named it as “smitlogs_WRITE”. This event will give you information about whether any user and even the root user ties to edit the smit.log file or update this file using smitty tool.
• Classes
Classes define group of events. A system administrator can group logically similar events in a class. For example both USER_SU & Password_Change events are related to “general” class of events. Class names are arbitrary and you can use any class name for certain group of events.
• Objects
Auditing objects means system configuration files as well as kernel related objects present in ODM (for example /etc/sendmail.cf and /etc/objrepos/SRCsubsys, both are example of audit objects). Read, write & execution activities related to these audit objects can easily be audited through AIX auditing.
• Users:
It is the responsibility of system administrator to identify the users who are to be audited for specific group of events (or so called classes). You can audit one or more class per user. For instance /etc/security/audit/config file may contain the following stanza.
user:
jack= general,cron,tcpip.
2.0 Data collection Modes
One of the most important configuration aspects of AIX auditing is the data collection mode in which it is operating. This data collection mode, basically describes the way in which data is collected by AIX auditing subsystems for analysis by system and security administrators.
There are two basic types of data collection by AIX auditing. One is named as binary mode and other is known as stream mode.
2.1 Binary Mode Configuration
While AIX auditing daemon operates in binary mode, audit events are recorded in two BIN files alternatively. This recording in binary files is temporary and finally data is appended into one single trail file. It must be noted down as the data format is binary so operating system commands like vi, pg or more can not be used to read the data directly from these files. Instead “auditcat“command is used to read the data from these files.
In essence, binary mode is used in scenarios, where long term audit data recording and analysis is required (for example by IT auditors or by IT security teams).
A schematic data flow in binary mode configuration of AIX audit subsystem is shown as below:
The alternating BIN mechanism (/auditfs/bin1 and /auditfs/bin2) is used to ensure that the audit susbsystem always has something to write to while the audit records are processed. When the audit subsystem switches to the other bin, it empties the first bin content to the /auditfs/trail file.
When time comes to switch the bin again, the first bin will be available. This mechanism ensures the decoupling of the storage and analysis of the data from the data generation.
Typically, the auditcat program is used to read the data from the bin that the kernel is not writing to at the moment. To make sure that the system never runs out of space for the audit trail (the output of the auditcat program), the freespace parameter can be specified in the /etc/security/audit/config file.
Let’s start with basic configuration steps for AIX auditing in binary mode. Configuration files of AIX auditing subsystem are located in /etc/security/audit directory. Main configuration file is “config”, which controls auditing subsystem basic behavior.
For enabling of binary mode, you just have to put “binmode = on” in config file.
It is important to note down that either of modes can be “ON” or both of them. It means that there is also a possibility of having both binary as well as stream modes “active” at same time.
It is always a good idea to create a separate filesystem for holding bin files as well as trail. It will eliminate any chance of filling up / filesystem or any other filesystem (as size of these files is usually large due to enormous number of records coming all together).
I therefore created a filesystem /auditfs and then configured auditing subsystem configuration file as shown below:
/home/root> more /etc/security/audit/config
------------------------------------------------------------------------------------------------------------
start:
binmode = on
streammode=off
bin:
trail = /auditfs/trail
bin1 = /auditfs/bin1
bin2 = /auditfs/bin2
binsize = 10240
cmds = /etc/security/audit/bincmds
freespace = 65536
stream:
cmds = /etc/security/audit/streamcmds
classes:
general =USER_SU,PASSWORD_Change
objects=S_ENVIRON_WRITE,S_GROUP_WRITE,S_LIMITS_WRITE,S_LOGIN_WRITE,S_PASSWD_READ,S_PASSWD_WRITE,S_USER_WRITE,AUD_CONFIG_WRITE
users:
root = general
-------------------------------------------------------------------------------------------------
As shown in this relatively simple configuration of auditing daemon, it is obvious that bin1 and bin2 parameter describes location of binary files while trail parameter specifies location of audit trail record.
The other two important parameters are binsize and cmd;”binsize” parameter specifies size of temporary files in bytes, before switching to the other binary temporary file while “cmd” specifies location of backend command for the BIN mode of AIX auditing subsystem. It is very important to note down that this backend program ( bincmds) is nothing more than usage of auditcat command as follows
/usr/sbin/auditcat -p -o $trail $bin
Events to be audited can be grouped in “classes” stanza of config file. As shown above, class named as “general” comprises of events related to user security (like switch user and password change) and then root user is configured to be audited against this class of events. It should be noted down that you can create classes of you own choice with arbitrary names and then use these classes to group predefined events as well as customized events.
Another important stanza of this configuration file is “objects”. In AIX auditing environment, auditing objects means system configuration files as well as kennel related objects present in ODM (for instance /etc/sendmail.cf & /etc/objrepos/SRCsubsys are both example of audit objects). Read, write & Execution of a file kernel object can be audited through audit objects. You can either specify objects in config file under the stanza of “objects” for relatively simpler configurations but for more complex auditing configurations you can specify all objects of your need in “object” file also, which is present in /etc/security/audit directory.
2.2 Stream Mode Configuration
The stream mode writes the audit records in a circular buffer in memory. The root user or audit group member user can continuously view the records from stream.out file by using vi, more or pg commands.
Structurally, STREAM mode writes the audit records to a circular buffer that can be read
by a /dev/audit device file, as shown in Figure 2-3 on page 10. When the kernel reaches the end of the buffer, it simply wraps to the beginning.
In stream mode, auditstream command is used to read the /dev/audit audit device file. On the other hand, the auditselect command can be used to select only those events in which system administrators are interested. This objective can be achieved by –c flag of auditselect command.
It is obvious Stream mode of AIX auditing is more likely to be used in those environments where system and security administrators are interested in monitoring real time data from audit events and generating some kinds of traps for more crucial security related events.
In the STREAM mode, the kernel writes records into a circular buffer. When the kernel reaches the end of the buffer, it simply wraps to the beginning. Processes read the information through a pseudo-device called /dev/audit. When a process opens this device, a channel is created for that process. Optionally, the events to be read on the channel can be specified as a list of audit classes. See the following figure for an illustration of audit STREAM mode:
As it is obvious, stream mode provides system and security administrators to have a real monitoring on audit events. This mode is therefore useful for the environments where auditors want to have continuous monitoring so that in case of any potential security breach or intruder attack, system and security administrators get notified immediately. Another use is to create a trail that is written immediately, preventing any possible tampering with the audit trail, as is possible if the trail is stored on some writable media.
Yet another method to use the STREAM mode is to write the audit stream into a program that stores the audit information on a remote system, which allows central near-time processing, while at the same time protecting the audit information from tampering at the originating host.
Basic stream mode configuration can be achieved by modifying /etc/security/config file as shown below.
/home/root> more /etc/security/audit/config
-----------------------------------------------------------------------------------------------------------
start:
binmode = off
streammode=on
bin:
trail = /auditfs/trail
bin1 = /auditfs/bin1
bin2 = /auditfs/bin2
binsize = 10240
cmds = /etc/security/audit/bincmds
freespace = 65536
stream:
cmds = /etc/security/audit/streamcmds
classes:
genuser=USER_SU,PASSWORD_Change
objects=S_ENVIRON_WRITE,S_GROUP_WRITE,S_LIMITS_WRITE,S_LOGIN_WRITE,S_PASSWD_READ,S_PASSWD_WRITE,S_USER_WRITE,AUD_CONFIG_WR
users:
root = general
-------------------------------------------------------------------------------------------------
As shown above, there are hardly any significant changes to configure stream mode as compared to what we did to configure binary mode, except we “on” stream mode and switch “off” binary mode.
It is important that streamcmd is nothing but a combination of auditstream and auditpr command:
/home/root> more /etc/security/audit/streamcmds
The output would be as follows:
/usr/sbin/auditstream | auditpr > /audit/stream.out &
As a result , whenever you start stream mode of AIX audit subsystem , by default you will start getting audit records in stream.out file. You can use either vi or cat or tail command to view real time audit records generated by auditing subsystem.
# tail -f /audit/stream.out
event login status time command
--------------- -------- -------- --------- ---------
S_NOTAUTH_READ root OK Thu May 24 14:07:05 2007 cat
S_NOTAUTH_READ root OK Thu May 24 14:07:05 2007 cat
FILE_Unlink root OK Thu May 24 14:07:09 2007 vi
S_NOTAUTH_READ root OK Thu May 24 14:07:09 2007 vi
S_NOTAUTH_READ root OK Thu May 24 14:07:09 2007 vi
S_NOTAUTH_READ root OK Thu May 24 14:07:09 2007 vi
3.0 Customized auditing output presentations
One of the most important aspect of AIX auditing subsystem is that juts like all other auditing subsystems of various other operating system like windows, Linux or Sun Solaris , it generates lot of audit records. As a result , output data coming from auditing is huge.
Best approach to overcome this problem is to properly configure auditing subsystem so that not all events and users should be audited. Instead it is always a good idea to identify critical system files, events and commands to be monitored and restrict this monitoring to few users only.
Finally, you can also restrict output audit records to certain specific auditable events. For example , in above scenerio where I configured auditing for “smit_READ” and “smit_WRITE”, there are other corresponding audit events appear in audit log like “FILE_Unlink”. There is provision of offloading these undesirable events so that information gathered through AIX auditing subsystem will be more specific.
There are several commands present, a combination of which can be used for this objective. Main command is “auditpr” which is used for formatting and display of audit records.
For example , after you setup binary or stream mode of audit subsystem and start audit subsystem ( using audit start command) you can use following “auditpr” command to display or read all audit records .
/home/root> auditpr -hhelpPRtTc -v | more
In stream mode, you can also use a combination of commands to display only selected events related audit records. There is “auditstream” command which stream out incoming data from audit subsystem. The resulted stream data can be inputted to “auditselect “command to select only specific audit events and discard remaining ones.
For instance, as in above example of auditing “smit_WRITE” event, if system administrators wants to collect related data to this specific event in a file “criticalwrites.out” then following combination of commands can be used as follows:
/home/root> /usr/sbin/auditstream | /usr/sbin/auditselect -e "event == smit_Write" | auditpr -hhelpPRtTc -v >> /home/root/criticalwrites.out
Similarly if system administrator wants to watch this critical event on system console , he can modify the above command as follows
/home/root> /usr/sbin/auditstream | /usr/sbin/auditselect -e "event == smit_Write" | auditpr -hhelpPRtTc -v > /dev/console &
4.0 Critical system events auditing configuration
Now we proceed to some relatively complex audit configuration to monitor some critical activities on AIX servers. We will create some customized audit events for this purpose, with our own defined objects to fulfill our requirements.
Event-A Monitoring changes in a specific file
There are certain important files which are always under root user ownership and system administrator with root password can easily play with these files. Traditionally, system administrator being super user for the UNIX operating system has full privileges so he can execute any commands through command line or through smitty menus and then delete the entries from corresponding log files (root’s shell history file and /smit.log files) This has remain a point of concern for IT auditors so here we proceed to configure auditing against any attempt to modify these critical files.
Let’s start with /smit.log file as an object for AIX 5L auditing. Same configuration steps can be used for monitoring root user shell history file.
First of all, I had to configure /smit.log file as an auditing object to be monitored by AIX auditing subsystem. For this purpose, I added following stanza in /etc/security/audit/objects file:
/smit.log:
r = "smitlogs_READ"
w = "smitlogs_WRITE"
Next, I added following entries in /etc/security/audit/config file as follows
classes:
readwrite = smitlogs_smitlogs_WRITE
It has added an event class of readwrite and finally we have to assign users of our interest (whom we want to audit against read write attempts to smit.log file). This again done in same file (/etc/security/audit/config)
users:
root = general, readwrite
jack = readwrite
thomas = readwrite
john = readwrite
Now in which data collection mode, we want to monitor this specific event, it really depends upon our own choice. Let’s assume, we want to monitor this specific event in stream mode , so we just “off” the binary mode in config file ( as described above) and “On” stream mode , start audit subsystem using “ audit start” command .
Because the data collection is enabled in STREAM mode, data collection can be started by running the following command:
# /usr/sbin/auditstream | auditpr -hhelpPRtTc -v
Optionally , as described in section 3.0 , you can combine auditselect command to select and display only specific events
#/usr/sbin/auditstream | /usr/sbin/auditselect -e "event == smit_Write" | auditpr -hhelpPRtTc -v
Or redirecting it to a file for later review by auditors
# usr/sbin/auditstream | /usr/sbin/auditselect -e "event == smit_Write" | auditpr -hhelpPRtTc –v >> /auditfs/criticalevents.out
The auditing results are written in the /auditfs/criticalevents.out file, which can be monitored in real time to keep track of the read and write operations.
If you use first option, typical output would look like below.
Listing 1. Output file—Data collection in STREAM mode
# tail -f /auditfs/criticalevents.out
event login status time command
--------------- -------- -------- --------- ---------
smit_READ root OK Thu May 24 14:07:05 2007 cat
smit_READ root OK Thu May 24 14:07:05 2007 cat
smit_READ root OK Thu May 24 14:07:09 2007 vi
smit_READ root OK Thu May 24 14:07:09 2007 vi
smit_READ root OK Thu May 24 14:07:09 2007 vi
smit_WRITE root OK Thu May 24 14:07:13 2007 vi
FILE_Unlink root OK Thu May 24 14:07:13 2007 vi
FILE_Unlink root OK Thu May 24 14:07:20 2007 vi
The interpretation of the output file is relatively simple as it shows that first of all root user opened the smit.log file using cat command and then by using vi command until the first write was done by root user on Thursday, May 24th, at 14:07:13, after which several operating systems events instances of “FILE_unlink” which updates file related information like inodes and file sizes accordingly.
If data collection in BIN mode is enabled, you can start the data collection by executing the following command:
# /usr/sbin/auditpr -v < /auditfs/trail > /auditfs/audit.out
This command writes the results of auditing to the /audit.out file, which can be monitored in real time as well. A sample is shown below:
Listing 2. Output file—Data collection in BIN mode
# vi /audit.out
"/audit.out" 30 lines, 2012 characters
event login status time command
-------- -------- ----------- ------------- --------------
smit_READ root OK Thu May 24 15:07:27 2007 cat
smit_READ root OK Thu May 24 15:07:27 2007 cat
FILE_Unlink root OK Thu May 24 15:07:32 2007 vi
filename /var/tmp/Ex21778
smit_WRITE root OK Thu May 24 15:07:37 2007 vi
FILE_Unlink root OK Thu May 24 15:07:37 2007 vi
Event-B Monitoring execution of specific command
Another event which may be of very much interest for security administrators and IT auditors is the event of execution of any specific command on system (like rmdev, cfgmgr or even simple rm command).
If you review /etc/security/audit/events, you can find many commands which generate many system related events. For example, you can find a stanza for rmdev command which generates three events namely “DEV_Stop”, “DEV_Unconfigure” and “DEV_Remove”. It means that when system administrator executes “rmdev” command, it will generate all these events.
Now if you want to audit execution of “rmdev” command by root user , first of all you have to add following entry in objects file
I first of all added following stanza in /etc/security/audit/objects file:
/usr/sbin/rmdev
x = "DEV_Remove"
Next, I added following entries in /etc/security/audit/config file as follows
classes:
commandexec = DEV_Remove
Finally I assigned this class of DEV_Remove to one of my developer user so that whenever he executes rm command on systems, security administrators get notified
users:
root = readwrite, commandexec
You can add some more commands to same class. For example to add “rmlv” command in same class, I have to add another object like follows
/usr/sbin/rmlv:
x = " LVM_DeleteLV "
and then add this auditable action to event class of “commandexec”
classes:
commandexec = DEV_Remove,LVM_DeleteLV
Now as event class of “commandexec” has already been assigned to root user, so you can start auditing of execution of rmlv command by shutting down and then restarting audit daemon.
Finally monitoring of this specific event can be done by
#/home/root>/usr/sbin/auditstream | /usr/sbin/auditselect -e "event==LVM_DeleteLV"| auditpr -hhelpPRtTc -v
Event-C Monitoring users related activities
Su and password change events are very critical events on UNIX operating system , especially with reference to root user. These events are , therefore by default monitored by AIX auditing subsystem.
In my scenario, config file already included these events. The events of “USER_SU” and “PASSWORD_Change” are already member of general class of events and are assigned to root user. So whenever root user switch to any other user , using su command or change his password using passwd command , it is being audited by AIX auditing subsystem.
For auditing any attempt by root user (or any other user) to change characteristics of any other user, i simply added following line in objects file , followed by changes in config file ( both mentioned below)
Added new entry in Objects File
/usr/bin/chuser:
X = “USER_Change“
Added new entry in Config File
general = USER_SU,PASSWORD_Change,DEV_Remove,LVM_DeleteLV,USER_Change
Summary:
Auditing always being a powerful tool for ensuring systems security and integrity, has been point of interest for IT auditors since long time, however its strength can only be realized in real sense when system and IT security administrators use this tool in positive sense to enhance systems security. AIX auditing subsystem is no more exception and it can be used for improving corporate level security , however as it generates lot of logs and therefore may put processing load in production environment , therefore it is required to configure this subsystem properly and for auditing of specific events only.
References: AIX 5.3 Security Guide IBM -SC23-4907-03
AIX 5L Auditing and Accounting IBM Red Book
About Author: Khurram Shiraz is Technical Consultant at GBM, Kuwait. In his ten years of IT experience, he worked mainly with IBM technologies and products especially AIX, HACMP Clustering, Tivoli and IBM SAN/ NAS Storage. He also has worked with IBM Integrated Technology Services group. His area of expertise includes design and implementation of high availability and DR solutions based on pSeries, Linux and windows infrastructure. He can be reached at kshiraz12@hotmail.com
No comments:
Post a Comment