Linux Archives - Sergey Lysenko

Kaniko Transitions to Chainguard: What It Means for CI/CD

Sergey.Lysenko — Fri, 03 Oct 2025 08:25:47 +0000

For years, Kaniko has been a go-to tool for building container images inside CI/CD pipelines, especially in environments where Docker isn’t available. However, active development on Kaniko by Google has now come to an end.

The good news is that the project isn’t disappearing. Chainguard has stepped in, forking Kaniko and taking over its ongoing development. This move ensures that the tool remains available and continues to evolve, with a strong focus on modern security practices and supply chain integrity — areas where Chainguard has already established deep expertise.

From a DevOps perspective, this transition raises some interesting points. On the one hand, it’s reassuring to see a community-driven fork keep a widely used tool alive. On the other hand, it highlights the fragility of relying on single-vendor projects, especially when they underpin critical workflows in CI/CD systems.

This shift also opens the door for teams to re-evaluate their image build strategy. Should they continue with Kaniko under Chainguard’s stewardship, or consider alternatives like BuildKit, Tekton, or other container-native solutions?

The post Kaniko Transitions to Chainguard: What It Means for CI/CD appeared first on Sergey Lysenko.

How to Reveal instanceType in AWS Fargate

Sergey.Lysenko — Tue, 10 Dec 2024 09:36:21 +0000

AWS Fargate is a serverless compute service that abstracts the underlying infrastructure, including the instanceType. This can be challenging when you need to determine the exact type of instance your container is running on. However, there’s a straightforward way to uncover this information.

You can reveal the hidden instance type by accessing the container (or pod) and executing a specific command:

cat /sys/devices/virtual/dmi/id/product_name

Why This Works

The file /sys/devices/virtual/dmi/id/product_name contains hardware information provided via the Desktop Management Interface (DMI). Even in Fargate, this data is passed through the virtualization layer, making it accessible from within the container or pod.

The post How to Reveal instanceType in AWS Fargate appeared first on Sergey Lysenko.

Sync SFTP remote folders via CLI in Linux

Sergey.Lysenko — Thu, 17 Mar 2022 13:19:09 +0000

There are many utilities for synchronizing folders via SFTP on Linux, but most of them are visual (eg Filezilla). There are not so many good utilities for the command line. For example, there is the sftp utility, but it is not very convenient, you cannot synchronize the whole folder recursively using it. Therefore, here we will look at lftp, which works from the command line and allows you to do a lot of operations using the SFTP protocol.

Installing lftp is very easy. For example, for Ubuntu:

apt-get install -y lftp;

After installation, you can immediately use it. The logic of the program is such that we can write a script and run it. There at script will be a connection to the SFTP server and some action that we need. For example, let’s say we need to copy the entire folder from the local computer to the server via SFTP. Example of a command:

lftp sftp://username:password@sftp.example.com -p 22 -e 'set sftp:connect-program "ssh -o StrictHostKeyChecking=no -a -x -i /home/.ssh/yourkey.key"; mirror -eRv /var/from-folder/ /var/to-folder; quit;'

Here is a command to connect to an SFTP server using an ssh key. Commands for lftp are listed in the -e option separated by semicolons. After the connection, the contents of the /var/from-folder/ folder are copied from the local computer to the /var/to-folder folder on the server recursively (with all subfolders).

Note that to login with a key, the sftp:connect-program variable is first set, which contains the ssh connection command and path to the key. The StrictHostKeyChecking option has been added so that the server’s fingerprint will not be checked, it is assumed that we trust it.

Even if we use a key, the password must be specified: username:password. If not specified, lftp will ask for a password. If we authorize by key, we need to specify any string as a password.

The post Sync SFTP remote folders via CLI in Linux appeared first on Sergey Lysenko.

A simple way to copy emails from one account to another

Sergey.Lysenko — Fri, 18 Feb 2022 10:14:11 +0000

If you need to copy emails from one mailbox to another, even if they are located on different servers, you can use the imapsync utility. You just need to install and run it in the console with parameters. Naturally, you need to know the passwords for both mailboxes.

To install the imapsync on a Mac, you can use homebrew:

brew install imapsync

For installation on other operating systems, follow the instructions.

After installation, just run the command:

imapsync --host1  --user1  --password1  --host2  --user2  --password2  --subfolder2 "" --dry

This command will create a folder named “destination-folder” in the mailbox and copy all emails with all folders from the source mailbox to the specified folder.

The –dry option is needed to see what the command will do without doing anything. If you’re sure about the result, just remove –dry and run it again.

Synchronization can take a very long time, depending on the size of the mailbox. If you interrupt the process and start it later, the synchronization will continue from where you left off last time.

In general, the utility is very useful, and it has many other interesting parameters.

The post A simple way to copy emails from one account to another appeared first on Sergey Lysenko.

Fluentbit does not follow the logs

Sergey.Lysenko — Mon, 14 Feb 2022 14:47:16 +0000

Fluentbit can follow and parse logs and send them to different systems, for example, Newrelic or Elasticsearch. But you may notice, that for some reason it does not follow files or stops follow ones.

By default, the precompiled version of Fluentbit goes with inodes support. And if your logs are on the NFS storage, then you may see warnings:

[2021/12/16 06:29:19] [debug] [input:tail:tail.4] scan_blog add(): dismissed: /var/log/nginx/access.log, inode 2621611

That exactly points to something wrong with inodes. There is a known issue at Fluentbit, where you can find out the details.

To resolve the issue on NFS, you should recompile the Fluentbit without inotify support (FLB_INOTIFY parameter):

cmake -DFLB_INOTIFY=Off

I am using td-agent-bit, and here is my Dockerfile:

ARG FLB_VERSION=1.8.11

FROM debian:bullseye-slim as builder

ARG FLB_VERSION
ARG FLB_TARBALL=https://github.com/fluent/fluent-bit/archive/v$FLB_VERSION.tar.gz
ENV FLB_SOURCE $FLB_TARBALL
RUN mkdir -p /fluent-bit/bin /fluent-bit/etc /fluent-bit/log /tmp/fluent-bit-master/

ENV DEBIAN_FRONTEND noninteractive

RUN apt-get update && \
    apt-get install -y --no-install-recommends \
    build-essential \
    ca-certificates \
    curl \
    cmake \
    make \
    tar \
    libssl-dev \
    libsasl2-dev \
    pkg-config \
    libsystemd-dev \
    zlib1g-dev \
    libpq-dev \
    postgresql-server-dev-all \
    flex \
    bison \
    && curl -L -o "/tmp/fluent-bit.tar.gz" ${FLB_SOURCE} \
    && cd tmp/ && mkdir fluent-bit \
    && tar zxfv fluent-bit.tar.gz -C ./fluent-bit --strip-components=1 \
    && cd fluent-bit/build/ \
    && rm -rf /tmp/fluent-bit/build/*

WORKDIR /tmp/fluent-bit/build/
RUN cmake -DFLB_RELEASE=On \
          -DFLB_TD=On \
          -DFLB_INOTIFY=Off \
          -DFLB_TRACE=On \
          -DFLB_JEMALLOC=On \
          -DFLB_TLS=On \
          -DFLB_SHARED_LIB=On \
          -DFLB_EXAMPLES=Off \
          -DFLB_HTTP_SERVER=On \
          -DFLB_IN_SYSTEMD=On \
          -DFLB_OUT_KAFKA=On \
          -DFLB_OUT_PGSQL=On ..

RUN make -j $(getconf _NPROCESSORS_ONLN)
RUN install bin/td-agent-bit /fluent-bit/bin/

FROM newrelic/newrelic-fluentbit-output:1.12.1 as newrelic-fluentbit-plugin

FROM debian:bullseye-slim

MAINTAINER Sergey Lysenko

ARG FLB_VERSION

# Update certificates, install ed
RUN apt-get update && \
    apt-get install -y ca-certificates ed && \
    rm -rf /var/lib/apt/lists/*

# Add key for fluentbit
RUN apt-get update && \
    apt-get install -y gnupg && \
    apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 4FF8368B6EA0722A && \
    apt-get autoremove -y gnupg && \
    rm -rf /var/lib/apt/lists/*

# Install fluentbit
RUN echo "deb https://packages.fluentbit.io/debian/buster buster main" | tee /etc/apt/sources.list.d/fluentbit.list && \
    apt-get update && \
    apt-get install -y td-agent-bit=${FLB_VERSION} && \
    rm /opt/td-agent-bit/bin/td-agent-bit && \
    rm -rf /var/lib/apt/lists/*

# Add fluentbit user
RUN addgroup --system fluentbit && \
    useradd --no-log-init --create-home --home-dir /home/fluentbit --shell /bin/bash --uid 1234567 --system --gid root fluentbit && \
    usermod -a -G fluentbit fluentbit

COPY --from=builder /fluent-bit/bin/td-agent-bit /opt/td-agent-bit/bin/td-agent-bit
COPY --from=newrelic-fluentbit-plugin /fluent-bit/bin/out_newrelic.so /opt/td-agent-bit/bin/plugins/out_newrelic.so

RUN usermod -u 1234567 fluentbit

# Permissions
RUN dirs="/var/log/fluentbit /etc/td-agent-bit" \
    && mkdir -p $dirs \
    && chown -R fluentbit $dirs \
    && chgrp -R 0 $dirs \
    && chmod -R g=u $dirs \
    && chmod 664 /etc/passwd /etc/group

COPY config /etc/td-agent-bit
COPY entrypoint.sh /

USER fluentbit

ENTRYPOINT ["/entrypoint.sh"]

CMD ["/opt/td-agent-bit/bin/td-agent-bit", "-c", "/etc/td-agent-bit/td-agent-bit.conf"]

The post Fluentbit does not follow the logs appeared first on Sergey Lysenko.

How to check memory limit inside Docker container

Sergey.Lysenko — Tue, 11 Jan 2022 15:05:46 +0000

You should know the memory limit of your container (not the whole host) when packing applications there. For example and simplicity, let’s imagine you are creating customized redis, memcached or varnish image and you wish to know available memory when the container starts to set it as the memory limit for your application.

There are two options for it, you can choose one depending on cgroup version (v1 or v2), or check both.

/sys/fs/cgroup/memory/memory.limit_in_bytes
/sys/fs/cgroup/memory.max

So, here is the bash code snippet:

if [ -f "/sys/fs/cgroup/memory/memory.limit_in_bytes" ]; then
    container_memory_limit=$( cat /sys/fs/cgroup/memory/memory.limit_in_bytes )
elif [ -f "/sys/fs/cgroup/memory.max" ]; then
    container_memory_limit=$( cat /sys/fs/cgroup/memory.max )
fi

There in variable container_memory_limit will be a memory limit in bytes. Just in case you can also cast the result value to an integer using AWK (i like it):

container_memory_limit=$( awk -v memory_limit=$container_memory_limit 'BEGIN { printf "%0.0f\n", memory_limit }' )

The post How to check memory limit inside Docker container appeared first on Sergey Lysenko.

Show disk usage ignoring mounts

Sergey.Lysenko — Sat, 13 Nov 2021 12:32:39 +0000

If you wish to know how much space uses some dir in Linux, you can use du -hs /some-folder. But what if you have a lot of mixed mounts in that folder and you want to know how much exactly this folder consumes at root disk storage?

The solution is quite simple. You cannot filter mounts, but you can mount root partition again, and operate with the new mount:

# Go to a quiet place and create new mount for root
cd /var/tmp
mkdir fake-root
mount /dev/sda1 fake-root
# Here /dev/sda1 is your drive, it may differ

# Do some stuff

# Then done, cleanup
umount fake-root
rmdir fake-root

The post Show disk usage ignoring mounts appeared first on Sergey Lysenko.

Get count of files in a Linux directory recursively

Sergey.Lysenko — Thu, 28 Oct 2021 08:38:15 +0000

It is very simple:

for i in */ .*/ ; do 
    echo -n $i": " ; 
    (find "$i" -type f | wc -l) ; 
done

The post Get count of files in a Linux directory recursively appeared first on Sergey Lysenko.