PerfectScale is making right-sizing Kubernetes workloads a breeze. Cloud companies (including Doit) have a lot of their workloads in serverless technologies as well, AWS Lambda and ECS/Fargate, GCP Functions and CloudRun. The basic principle is very similar, looks at the CPU/Memory/GPU/Network usage of the workload, and adjust configuration as needed.

Cloud companies heavily rely on serverless and containerized workloads such as AWS Lambda, AWS ECS/Fargate, GCP Cloud Functions, and Cloud Run. These platforms promise automatic scaling and flexibility, but in reality, workload sizing presents significant challenges:

• Many companies allocate excessive resources (CPU, memory, GPU) out of fear of performance bottlenecks. For example: A function that needs 256MB of memory might be allocated 2GB just to be safe, leading to massive cost inefficiencies.
• If workloads lack the right resources, they experience latency spikes, throttling, or outright failures. For example: A Cloud Run service with too little CPU might struggle under load, delaying responses and causing timeouts.
• Cloud monitoring tools like CloudWatch, GCP Metrics, and Datadog provide metrics, but they don’t auto-tune workloads. Engineers must manually adjust configurations, requiring deep knowledge of workload behavior.
• GPU instances and AI workloads require precise tuning to avoid wasting expensive GPU cycles. For example: Allocating an entire A100 GPU for an inference task that could run on a fraction of the hardware.
• Default autoscalers react to simple CPU or memory thresholds but don’t optimize for cost vs. performance trade-offs. For example: AWS Lambda scales horizontally, but if each instance is over-provisioned, scaling still leads to unnecessary cloud spend.

By applying AI-driven right-sizing, we can continuously analyze workload behavior and automatically adjust resource allocation, leading to:

• Automatically optimizes CPU, memory, and GPU allocation based on real-time usage patterns. For example: A Cloud Run service that usually needs 500m CPU and 1GB RAM but experiences occasional spikes can be set to adjust dynamically instead of being permanently over-provisioned.
• Ensures just enough resources are allocated to meet performance SLAs without overpaying. For example: AWS Lambda functions are dynamically tuned to use the lowest memory setting that still completes execution within time constraints.
• Monitors workloads over time and adjusts allocations dynamically to avoid drift. For example: A workload that initially needed 2 vCPUs but later drops to 1 vCPU is automatically resized to reduce costs.
• Traditional autoscaling only scales horizontally, but this solution right-sizes individual instances, preventing waste before scaling. For example: AI inference workloads running on GPUs are allocated precise memory and GPU resources, minimizing wasted compute cycles.
• AI can learn from historical data to suggest better workload configurations before problems occur. Engineers no longer have to manually adjust configurations based on guesswork or periodic audits.

Current approach coping with these issues:

• Engineers adjust CPU/memory settings through trial and error.
• Defaulting to high allocations to prevent failures.
• Basic CPU/memory thresholds trigger scaling events, but they don’t prevent waste.
• Teams manually audit cloud bills and look for over-provisioned services, but by the time adjustments are made, money has already been wasted.
• Companies often pay for unnecessary resources just to be safe.
• Some workloads remain under-provisioned and struggle under load.
• Configurations don’t adjust dynamically as usage patterns change.

How would it work after implementing this solution?

• No more manual tuning; workloads are dynamically adjusted based on real-time behavior.
• Right-sizing prevents unnecessary spend while maintaining performance.
• Optimized workloads reduce latency and failure rates.
• AI ensures that every new instance is right-sized from the start.
• Instead of firefighting resource allocation, engineers focus on building features.

What must cloud users change in order to receive the full benefit of such technology?

• Connect Lambda, Cloud Run, ECS/Fargate, Kubernetes to the optimizer.
• Let AI analyze workloads and learn from usage patterns over time.
• Move away from static configurations and let the system dynamically right-size workloads.
• Instead of just setting CPU/memory manually, let AI balance cost and performance dynamically.

By removing inefficiencies in serverless and containerized workloads, this solution makes cloud infrastructure smarter, leaner, and more adaptive—giving companies a huge competitive edge in cloud cost management and performance optimization.