There are a lot of tools for debugging kernel and userspace programs in Linux. Most of them have performance impact and cannot easily be run in production environments. A few years ago, eBPF was developed, which provides the ability to trace the kernel and userspace with low overhead, without needing to recompile programs or load kernel modules.
Read more
Linux is a pleasure to use, and system administrators agree, especially on the last Friday in July, a.k.a. System Administrator Appreciation Day (a.k.a. SysAdmin Day, a.k.a. Sysmas).
In honor of Sysadmin Day, we’ve put together a list of our favorite features and Easter eggs in Linux. Enjoy! Read more
Original publication date: September 22, 2015.
The audit subsystem is used to raise the level of security in Linux systems. Although it doesn’t offer additional security per se, it’s used to retrieve detailed information on system events. This provides detailed information on system violations, which can be used to implement additional targeted security measures. We’ll be taking a deeper look at the audit subsystem in this article. Read more
In May 2016, the developers of Sysdig released Falco, a tool for detecting anomalous system behavior.
Falco consists of two main components: the sysdig_probe kernel module (which Sysdig also runs on) and the daemon for writing the information it collects to the disk.
Falco tracks applications according to user-defined rules, and if any anomalies are detected, it writes the information to a standard output, syslog, or user-defined file. in their blog, the developers jokingly call Falco “…a hybrid of snort, ossec and strace,” and position it as a simple IDS that puts almost no additional load on the system.
In previous posts, we looked at various issues related to monitoring, collecting, and saving metrics and different methods and tools for getting around them. In today’s article, we’ll be discussing yet another tool that simplifies these processes: Netdata.
Unlike other analytical tools, Netdata is designed for gathering and visualizing metrics in real time (and when necessary, a backend can be incorporated for gathering and saving collected metrics).
In our previous article, we explored problems tracing and profiling the Linux kernel.
Today, we’d like to revisit these issues and talk about an interesting kernel tracer, LTTng, which was developed by Canadian programmer and researcher Mathieu Desnoyers. With this tool, we can receive information on events that occur in both the kernel and user spaces.
There are a number of tools for analyzing events at the kernel level: SystemTap, ktap, Sysdig, LTTNG, etc., and you can find plenty of detailed articles and materials about these on the web.
You’ll find much less information on Linux’s native mechanism for tracing system events and retrieving/analyzing troubleshooting information. That would be ftrace, the first tracing tool added to the kernel, and this is what we’ll be looking at today. Read more
Today we’ll be continuing our post series on containerization mechanisms. In our last article on containerization, we talked about isolating processes using the namespaces mechanism. For containerization though, isolating resources isn’t enough. If we launch an application in an isolated environment, we should be sure it has been allocated enough resources and not use an inordinate amount, interrupting the rest of the system. For this task, the Linux kernel has a special mechanism, cgroups (short for control groups), which we will talk about today.
Containerization has become an increasingly relevant topic. There are already thousands, if not tens of thousands, of articles and posts written about popular solutions like LXC and Docker.
In today’s article, we’d like to discuss systemd-nspawn, a systemd component for creating isolated environments. Systemd is already a standard in the world of Linux and in light of this, it wouldn’t be unfounded to suggest that the potential for systemd-nspawn will significantly expand in the near future. For this reason, we think now would be a good time to better acquaint ourselves with this tool.
The initialization daemon systemd has become the de facto standard in modern Linux systems and is already used in many popular distributions: Debian, RHEL/CentOS, and Ubuntu (as of ver. 15.04). Compared to the traditional syslog, systemd offers an entirely different approach to logging.
At its base you’ll find centralization; the journal component collects all of the system messages (messages from the kernel and from various services and applications). In this case, there’s no need to configure log distribution, instead the applications can just write to stdout and stderr, and journal automatically saves these messages. This setup is possible with Upstart, but Upstart saves all logs to a separate file, whereas systemd saves them in a binary base, greatly simplifying systemization and searches.
Read more