Today, we are diving deep into the two titans of Kubernetes configuration management: Helm and Kustomize. Both tools aim to solve the problem of environment-specific configurations, but they do so using fundamentally different philosophies.

Let’s break down their advantages, disadvantages, and how they perform across the complete deployment lifecycle.

The Helm Approach: Templating and Package Management

Helm is the “package manager for Kubernetes.” It uses Go templating to inject variables (values) into YAML manifests. You create a single Chart and deploy it across different environments by supplying different values.yaml files.

Advantages of Helm

• Powerful Logic and Control Flow: Helm allows you to use if/else statements, loops (range), and complex functions within your manifests. This means you can conditionally enable entire architectural components (like an embedded Redis for local, but an external AWS ElastiCache for production).
• Release Management: Helm tracks your deployments as “releases.” You can easily see the deployment history using helm history and rollback to a previous state instantly with helm rollback.
• The Chart Ecosystem: If you need to deploy third-party software (like Nginx Ingress, Prometheus, or Cert-Manager), Helm is the industry standard. Artifact Hub provides thousands of ready-to-use charts.
• All-in-One Packaging: A Helm Chart bundles everything your application needs into a single versioned artifact, making it highly portable.

Disadvantages of Helm

• The “Wall of Text” and Indentation Hell: Go templating inside YAML is notoriously difficult to read and debug. A missing space in an indentation logic block can break the entire deployment.
• Over-Engineering: For simple applications, creating a Helm chart with fully parameterized variables is often overkill.
• Drift Detection: Because Helm renders manifests client-side (or in CI/CD) and applies them, it can sometimes lose track of resources if they are manually modified in the cluster, though helm upgrade generally corrects this.

The Kustomize Approach: Patching and Overlays

Kustomize takes a fundamentally different path: No templates. Kustomize uses a concept of a base (your standard, raw YAML manifests) and overlays (environment-specific patches). It is natively built into kubectl (using kubectl apply -k).

Advantages of Kustomize

• Pure, Readable YAML: Since there are no Go templates, your base and overlays remain valid, readable YAML files. There is no cognitive load of trying to parse templating logic in your head.
• Native Integration: You don’t need an extra binary. Kustomize is built directly into kubectl, making it instantly available on any machine that interacts with a Kubernetes cluster.
• Declarative Simplicity: You explicitly define what needs to change. If the production environment needs 5 replicas instead of 1, you write a specific patch for the Deployment replica count.
• Easier Debugging: Running kustomize build outputs plain YAML. If something is wrong, you know exactly which patch caused it.

Disadvantages of Kustomize

• Lack of Package Management: Kustomize is strictly a configuration tool. It does not track releases, revisions, or provide easy rollbacks. Once applied, Kubernetes simply sees the updated state.
• Repetition in Drastically Different Environments: If your production environment is fundamentally different from your local environment (e.g., completely different sidecar containers, distinct routing logic), Kustomize overlays can become convoluted and repetitive.
• Limited Dynamic Injection: Passing dynamic variables from a CI/CD pipeline (like a Git commit hash for an image tag) requires using kustomize edit set image via bash scripts, which feels less elegant than Helm’s --set image.tag=$COMMIT.

Battle-Testing in Different Environments

How do these tools actually perform when moving code from a developer’s laptop to production?

1. Local Environment (Minikube / Kind / Docker Desktop)

• Kustomize: Shines here. Developers can run kubectl apply -k overlays/local instantly. It’s lightweight and doesn’t require setting up a complex values hierarchy.
• Helm: Can feel heavy for local development. Developers often struggle to debug complex templating logic just to get an app running on localhost.

2. Review Environments (Ephemeral CI/CD Deployments)

• Helm: Wins significantly. Ephemeral environments often require dynamic configurations (e.g., creating a unique namespace and Ingress host like pr-123.review.example.com). Helm handles this gracefully using --set ingress.host=....
• Kustomize: Requires sed/awk scripts or tools like envsubst to dynamically replace strings in the overlay files before running kustomize build, which introduces pipeline complexity.

3. Staging Environment

• Kustomize: Excellent for ensuring the staging environment is a near-exact replica of production. The staging overlay usually only differs from the production overlay in resource limits and DNS names.
• Helm: Works perfectly, provided your values-staging.yaml is meticulously maintained to mirror production logic.

4. Production Environment

• Helm: The release management is a lifesaver. If a production deployment fails, helm rollback provides immediate remediation. The ability to version charts guarantees that you are deploying exactly what was tested in staging.
• Kustomize: Requires a strict GitOps approach (using tools like ArgoCD or Flux). Since Kustomize itself doesn’t track state, you rely entirely on Git history to revert changes.

The Verdict: Which One Should You Choose?

The truth is, you don’t necessarily have to choose. The modern DevOps landscape often embraces a hybrid approach.

When to choose purely Helm: If you are building a complex application intended to be distributed to multiple customers across different infrastructure providers, Helm is mandatory. Its packaging and templating are unmatched.

When to choose purely Kustomize: If you manage in-house microservices, have a strong GitOps culture (using ArgoCD/Flux), and want to keep cognitive load low for developers, Kustomize is the cleanest path.

The Hybrid Approach (Best Practice): Many elite DevOps teams use Helm to package external dependencies (databases, message queues, ingress controllers) and Kustomize for internal application logic.

Furthermore, Helm 3.1+ supports Post-Rendering. You can use Helm to generate the base templates, and then pipe that output through Kustomize to apply specific, last-minute patches. This gives you the powerful packaging of Helm with the elegant, template-free patching of Kustomize.

Ultimately, both tools are exceptional. Evaluate your team’s familiarity with Go templates, your CI/CD pipeline structure, and your rollback requirements to make the right choice for your clusters.

The post Helm vs. Kustomize: Choosing the Right Tool for Kubernetes Environments appeared first on Sergey Lysenko.

Understanding the Problem

PHP-FPM is responsible for handling PHP requests. In a Kubernetes environment, it typically runs inside a container within a pod. Throttling occurs when the PHP-FPM process tries to consume more CPU resources than the Kubernetes scheduler allows it to use.

When a container exceeds its allocated CPU limits, Kubernetes (specifically, the Linux kernel’s Completely Fair Scheduler (CFS) bandwidth control) throttles its CPU usage. This means the container is artificially slowed down, preventing it from monopolizing the node’s resources. While this protects other pods on the same node, it severely impacts the performance of your PHP application.

Why Does This Happen in EKS?

Several factors contribute to PHP-FPM throttling in EKS:

1. Inadequate CPU Limits: The most common cause is setting CPU limits too low for the demands of your application. If a burst of traffic hits your PHP-FPM container, it quickly hits the ceiling and gets throttled.
2. Incorrect PHP-FPM Configuration: Settings like pm.max_children, pm.start_servers, and pm.min_spare_servers directly impact how many PHP-FPM processes are running. If these aren’t tuned to your workload and available resources, you can easily exhaust CPU allocations.
3. Noisy Neighbors: If other resource-intensive pods are running on the same EKS node, they might be competing for CPU cycles. Even if your PHP-FPM pod hasn’t reached its absolute limit, the overall node contention can lead to performance degradation that mimics throttling.

How to Identify Throttling

Before fixing the problem, you need to confirm it’s actually happening. You can monitor CPU throttling using metrics like:

• container_cpu_cfs_throttled_seconds_total: This Prometheus metric tracks the total time a container has been throttled.
• container_cpu_cfs_throttled_periods_total: This metric shows the number of periods where throttling occurred.

If these metrics show significant spikes or a steady increase, your PHP-FPM containers are definitely being throttled.

The Solution: Banishing the Throttle

Resolving PHP-FPM throttling requires a multi-pronged approach:

1. Optimize Resource Requests and Limits

The first step is to accurately define the CPU resources your PHP-FPM containers need.

• Set Realistic Requests: The requests value ensures the pod is scheduled on a node with enough available CPU. Set this to the baseline CPU usage of your application during normal operation.
• Carefully Tune Limits: The limits value is the hard ceiling. If you set it too low, you’ll get throttled during spikes. If you set it too high (or remove it entirely), you risk a runaway process crashing the entire node. A common strategy is to set the CPU limit significantly higher than the request (e.g., Request: 200m, Limit: 1000m) to allow for bursts, but monitor closely.

Example Pod Spec snippet:

resources:
  requests:
    cpu: "250m"
    memory: "512Mi"
  limits:
    cpu: "1000m"
    memory: "1Gi"

2. Fine-Tune PHP-FPM Configuration

The PHP-FPM process manager needs to be configured to handle your traffic volume efficiently without spawning unnecessary processes that consume CPU.

• Process Manager (pm): Choose the right process manager. dynamic is usually the best choice for variable workloads, as it scales processes up and down based on demand. ondemand is good for very low-traffic sites, while static might be appropriate for highly predictable, high-traffic applications.
• pm.max_children: This is the most crucial setting. It defines the absolute maximum number of PHP-FPM processes that can exist simultaneously. Set this high enough to handle your peak concurrent requests, but low enough that the combined CPU usage of all these processes doesn’t exceed your container’s CPU limit. Calculate this carefully based on the average memory/CPU consumption of a single PHP request.
• pm.start_servers, pm.min_spare_servers, pm.max_spare_servers: If using the dynamic process manager, tune these to ensure you have enough idle processes ready to handle sudden spikes, but not so many that they waste resources when idle.

3. Horizontal Pod Autoscaling (HPA)

Instead of relying on a single large pod with high CPU limits, it’s often more effective to use Kubernetes Horizontal Pod Autoscaling (HPA).

HPA automatically increases the number of PHP-FPM pods when CPU utilization (or another metric) reaches a certain threshold. This distributes the load across multiple nodes, preventing any single pod from being throttled.

Example HPA configuration:

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: php-fpm-hpa
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: php-fpm-deployment
  minReplicas: 2
  maxReplicas: 10
  metrics:
  - type: Resource
    resource:
      name: cpu
      target:
        type: Utilization
        averageUtilization: 70

4. Optimize Your Application Code

Often overlooked, the most effective way to reduce CPU usage is to optimize the PHP application itself.

• Database Queries: Inefficient database queries are a major CPU drain. Optimize queries, add indexes, and use caching mechanisms (like Redis or Memcached).
• Code Profiling: Use tools like Xdebug or Blackfire to identify CPU-intensive bottlenecks in your code.
• Opcache: Ensure PHP Opcache is enabled and properly configured. This drastically reduces the CPU overhead of compiling PHP scripts on every request.

Conclusion

PHP-FPM throttling in AWS EKS is a manageable challenge. By understanding the interaction between Kubernetes resource limits and PHP-FPM process management, and by implementing proper autoscaling and code optimization, you can ensure your PHP applications run smoothly and efficiently, no matter how much traffic they receive.

The post Combating PHP-FPM Throttling in AWS EKS appeared first on Sergey Lysenko.

If you want to view only your AWS costs in New Relic without leaking unnecessary infrastructure data, you need a strict, custom IAM approach using API Polling. Here is the step-by-step guide to setting it up securely.

Step 1: Get Your New Relic Identifiers

First, we need to generate unique connection IDs from New Relic.

1. Log into your New Relic account and navigate to Infrastructure -> AWS.
2. Click Add an AWS account and choose the API polling method.
3. Keep this tab open. You will see an Account ID and an External ID. You need both for the next step.

Step 2: Create a Strict AWS IAM Policy

We will create a policy that restricts New Relic to only reading Cost Explorer and AWS Budgets data.

1. Open your AWS Management Console and go to the IAM dashboard.
2. Navigate to Policies and click Create policy.
3. Switch to the JSON tab and paste the following exact configuration:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "AllowNewRelicBillingOnly",
      "Effect": "Allow",
      "Action": [
        "budgets:ViewBudget",
        "ce:GetCostAndUsage",
        "ce:GetCostForecast",
        "ce:GetReservationUtilization",
        "ce:GetDimensionValues",
        "ce:GetTags"
      ],
      "Resource": "*"
    }
  ]
}

4. Click through to the end, name the policy NewRelicBillingOnlyPolicy, and save it.

Step 3: Create the IAM Role

Now, attach this policy to a dedicated role for New Relic.

1. In the AWS IAM console, go to Roles and click Create role.
2. Select AWS account as the trusted entity, then choose Another AWS account.
3. Paste the Account ID you got from New Relic in Step 1.
4. Check the Require external ID box and paste the External ID from New Relic.
5. Click Next, search for your newly created NewRelicBillingOnlyPolicy, and select it.
6. Name the role NewRelicBillingIntegrationRole and click Create role.
7. Open the role you just created and copy its ARN (Amazon Resource Name).

Step 4: Finalize the New Relic Setup

1. Go back to your open New Relic tab.
2. In the AWS account name field, enter a clear alias like AWS-Billing-Only so your team knows exactly what this integration does.
3. Paste your Role ARN.
4. On the next screen, uncheck every service except for Billing.
5. Save the integration.

Step 5: Verify Your Data

AWS Cost Explorer data is not real-time; it updates a few times a day. Wait about 15-30 minutes, then head to the New Relic Query Builder and run this NRQL query to see your total spend for the current month:

SELECT max(provider.estimatedCharges.Maximum) FROM FinanceSample SINCE 1 month ago TIMESERIES 1 day

If you see a rising chart, your cost data is successfully flowing into New Relic without exposing your sensitive infrastructure metrics!

The post How to Safely Send AWS Billing Data to New Relic (Cost Data Only) appeared first on Sergey Lysenko.

The Problem with Third-party GTM

A classic GTM integration looks like this:

https://www.googletagmanager.com/gtm.js?id=GTM-XXXX

This approach has several practical issues:

• Ad-blockers often block googletagmanager.com
• Browsers increasingly restrict third-party cookies
• Lower data quality (missing page views, sessions, events)
• Compliance complications (GDPR, consent management)

In short: your analytics slowly becomes blind, especially on modern browsers.

Why First-party GTM Helps

With first-party GTM:

• GTM is loaded from your own domain
• Requests look like regular site traffic
• Much harder to block
• Better data completeness
• Cleaner GDPR story (still needs consent, but more predictable)

Instead of a Google domain, the browser sees:

https://www.example.com/first-party-gtm/

High-level architecture

Browser
  |
  |  GET /first-party-gtm/?id=GTM-XXXX
  v
Nginx (reverse proxy)
  |
  |  proxy_pass
  v
googletagmanager.com

Example: Nginx Configuration

location /first-party-gtm/ {
        proxy_pass https://$gtm_id.fps.goog;
        # Since we are using host as variable, we need resolver
        resolver 1.1.1.1 8.8.8.8 valid=300s;
        resolver_timeout 5s;

        proxy_ssl_name $gtm_id.fps.goog;
        proxy_ssl_server_name on;

        proxy_set_header Host $gtm_id.fps.goog;

        # Forward client-related headers
        proxy_set_header X-Real-IP              $remote_addr;
        proxy_set_header X-Forwarded-For        $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto      $scheme;
        proxy_set_header X-Forwarded-Host       $host;
        proxy_set_header X-Forwarded-Uri        $request_uri;
        proxy_set_header X-Forwarded-Country    $geoip2_data_country_code;
        proxy_set_header X-Forwarded-Region     $geoip2_data_region_code;

        # Do not cache Google tag responses on Nginx level
        proxy_no_cache      1;
        proxy_cache_bypass  1;
}

Important Notes

• Consent is still required (this is not a GDPR workaround)
• Verify with DevTools → Network that requests hit your domain
• Monitor errors carefully after rollout
• Roll out gradually if possible

Result

After switching to first-party GTM we observed:

• Fewer blocked requests
• More stable analytics data
• Cleaner observability in logs
• Better long-term compatibility with browser privacy changes

The post First-party GTM: Why We Moved Away from Third-party Analytics appeared first on Sergey Lysenko.

The good news is that the project isn’t disappearing. Chainguard has stepped in, forking Kaniko and taking over its ongoing development. This move ensures that the tool remains available and continues to evolve, with a strong focus on modern security practices and supply chain integrity — areas where Chainguard has already established deep expertise.

From a DevOps perspective, this transition raises some interesting points. On the one hand, it’s reassuring to see a community-driven fork keep a widely used tool alive. On the other hand, it highlights the fragility of relying on single-vendor projects, especially when they underpin critical workflows in CI/CD systems.

This shift also opens the door for teams to re-evaluate their image build strategy. Should they continue with Kaniko under Chainguard’s stewardship, or consider alternatives like BuildKit, Tekton, or other container-native solutions?

The post Kaniko Transitions to Chainguard: What It Means for CI/CD appeared first on Sergey Lysenko.

One of the simplest ways to achieve this is by using crane, a command-line tool from Google’s go-containerregistry project.

crane index append \
  -m reg.example.com/project/app:1.0.0-amd64 \
  -m reg.example.com/project/app:1.0.0-arm64 \
  -t reg.example.com/project/app:1.0.0 -v

Here’s what happens step by step:

1. Two platform-specific images already exist in the registry: one built for amd64, the other for arm64.
2. The -m flags specify the manifests to include in the new index.
3. The -t flag defines the final multi-arch tag (1.0.0 in this case).
4. The -v flag enables verbose output, useful for debugging.

After running this command, pulling reg.example.com/project/app:1.0.0 will automatically serve the correct architecture-specific image depending on the client’s platform.

Why This Matters

• Seamless developer experience: Users don’t need to worry about which tag matches their machine.
• Kubernetes compatibility: Clusters with mixed node types (x86 and ARM) can run the same deployment manifest.
• Future-proofing: ARM-based infrastructure is growing rapidly (e.g., AWS Graviton, Apple Silicon), so multi-arch support is becoming essential.

Final Thoughts

Using crane index append is a lightweight yet powerful way to unify platform-specific images under a single tag. It fits naturally into CI/CD pipelines and ensures your container images are portable, modern, and ready for diverse runtime environments.

The post Building Multi-Architecture Images with crane index append appeared first on Sergey Lysenko.

You can reveal the hidden instance type by accessing the container (or pod) and executing a specific command:

cat /sys/devices/virtual/dmi/id/product_name

Why This Works

The file /sys/devices/virtual/dmi/id/product_name contains hardware information provided via the Desktop Management Interface (DMI). Even in Fargate, this data is passed through the virtualization layer, making it accessible from within the container or pod.

The post How to Reveal instanceType in AWS Fargate appeared first on Sergey Lysenko.

1. Time to Live (TTL):
   • Definition: TTL is the duration for which a cached object is considered fresh and can be served from the cache without checking the origin server.
   • Usage: When a request is made, the caching system checks if the object is still within its TTL. If yes, it serves the cached version; otherwise, it fetches a fresh copy from the origin server.
2. Grace:
   • Definition: Grace is an extension of the TTL concept. It represents the additional time during which a cached object can be served even after its TTL has expired, while the caching system attempts to fetch a fresh copy from the origin server.
   • Usage: If a cached object’s TTL has expired but it is still within the grace period, the caching system may serve it while initiating a background request to the origin server to refresh the content.
3. Keep:
   • Definition: Keep is another extension of the TTL concept, specifying the maximum time a cached object can be retained in the cache, irrespective of whether the TTL has expired.
   • Usage: Keep allows the caching system to retain objects in the cache for a longer duration, even if the TTL has expired. It is useful in scenarios where you want to keep certain objects in the cache for a fixed period, regardless of their freshness.

In summary:

• TTL: Specifies how long an object remains valid in the cache.
• Grace: Represents the additional time during which an expired object can still be served while a background fetch is attempted.
• Keep: Defines the maximum duration an object can be retained in the cache, regardless of its TTL.

These parameters provide flexibility in balancing the need for serving fresh content and minimizing the load on the origin server by intelligently managing cached objects. Configuring TTL, grace, and keep values requires consideration of the specific requirements and characteristics of the cached content.

The post Understanding the lifetime of Varnish cached objects: TTL, grace, and keep appeared first on Sergey Lysenko.

Understanding JIT in PHP 8: Let’s demystify the concept of JIT compilation in PHP 8. This powerful feature dynamically transforms code into machine instructions at runtime, ushering in a new era of performance optimization. Prepare to unlock the astonishing potential of JIT and witness its significant impact on your PHP 8 projects.

The Power Unleashed: With the introduction of JIT in PHP 8, developers experience remarkable performance gains. The ability to generate optimized machine code on the fly eliminates the need for repetitive interpretation.

Consider a scenario where a PHP script needs to execute a computationally intensive task multiple times. Without JIT, the interpreter would re-analyze and recompile the code during each execution, resulting in unnecessary overhead. However, with JIT, the initial compilation is followed by the generation of highly optimized machine code, leading to faster subsequent executions. This groundbreaking shift can yield performance improvements of up to 30% or more!

Real-World Applications: Let’s delve into real-world use cases where the influence of JIT in PHP 8 becomes evident:

1. E-commerce Websites: Imagine an e-commerce platform with thousands of concurrent users. JIT compilation in PHP 8 can significantly enhance response times, ensuring seamless shopping experiences even during peak traffic periods. This boost in performance directly translates to increased customer satisfaction and improved conversion rates.
2. Data-Intensive Applications: JIT’s impact extends to data-intensive applications as well. Complex algorithms and calculations can be executed with astonishing speed, enabling businesses to process large datasets more efficiently. This enhanced performance empowers data-driven decision-making, leading to optimized operations and improved productivity.

Conclusion: As we conclude our exploration of JIT’s influence on PHP 8’s performance, we can’t help but appreciate the transformative power this feature brings to the realm of web development. With JIT compilation, PHP 8 unlocks new levels of efficiency, efficiency that translates into tangible benefits for businesses and end-users alike. Embrace the possibilities, harness the power, and witness the remarkable improvements in your PHP 8 projects.

The post The Impact of JIT on Performance in PHP 8: Unleashing the True Potential appeared first on Sergey Lysenko.

First of all I have to warn you about using the optimised directory structure. At one single folder you should keep not more than 100-200 files.

For example, this is very bad for performance:

/folder/0001.txt
/folder/0002.txt
/folder/0003.txt
/folder/0004.txt
***
/folder/9999.txt

If there will be thousands of files in one single folder, the performance will be very poor. It could take minutes to read files from that plain directory structure.

It should be like that:

/folder/a/0001.txt
/folder/a/0002.txt
/folder/b/0003.txt
/folder/b/0004.txt

As you can see, the files are grouped by subfolders (the same way Magento uses at media folder). At each subfolder there are minimum amount of files. The reading of the files will be very fast.

So, I can offer 3 variants of using (s)FTP service:

1) Build (s)ftp service yourselves

It can be a simple AWS instance with ftp/sftp enabled and storage volume attached. Here we can make all the functionality as we wish, including security optimisations. The build and setup of our own sftp could take up to 2 days of work and it will cost approximately 30-50 USD/month.

2) Use AWS Transfer Family

It is fully managed professional service, but very and very expensive. The costs start from 400 USD/month.

3) Use another managed ftp services

• https://hostedftp.com/
• https://sftptogo.com/
• https://www.maytech.net/features/sftp
  …etc.
  They are cheaper than AWS, but it seems they are less functional. We do not need to maintain them, it is the advantage. But we have absolutely no control on them, and will use it as a blackbox. The costs start from 40 USD/month.

As a result, I suggest to deploy own (s)ftp service (1-st point), because it has acceptable cost and we can do with it whatever we wish.

But if you have not enough skills, you should of course use managed ftp service (3-rd point).

If you need any help in setting up sFTP, you can contact me.

The post SFTP service for Magento appeared first on Sergey Lysenko.