Scalable Resource Management with HikariCP and Selenium Grid
Modern software systems are expected to scale efficiently while maintaining predictable performance under load. Two areas that frequently become bottlenecks in backend and automation-heavy applications are database connectivity and browser-based testing infrastructure.
This article demonstrates how to address both concerns using HikariCP for efficient database connection pooling and Selenium Grid for scalable browser automation. Together, these tools provide a strong foundation for high-throughput applications and parallelized test execution.
Why Resource Management Matters
Every database connection and browser instance consumes memory, CPU, and network resources. When these resources are repeatedly created and destroyed, applications suffer from:
- Increased latency
- Resource contention and exhaustion
- Reduced throughput under load
- Slower and less reliable test execution
The solution is controlled reuse:
- Database connections should be pooled and reused.
- Browser instances should be centrally managed and distributed.
Database Connection Pooling with HikariCP
The Real Cost of Database Connections
Opening a database connection is an expensive operation. Beyond the initial connection call, it involves network handshakes, authentication, optional TLS negotiation, and server-side resource allocation. Under concurrent load, repeatedly opening and closing connections can overwhelm both the application and the database server.
Connection pooling mitigates this by maintaining a pool of established, ready-to-use connections that can be reused across requests.
Why HikariCP?
HikariCP is widely adopted in production systems due to its low overhead, predictable performance, and operational simplicity. It is also the default connection pool used by Spring Boot.
Key characteristics include:
- Minimal internal locking
- Fast connection acquisition
- Clear, well-documented configuration options
- Proven reliability under high concurrency
Running MySQL with Docker Compose
Docker Compose provides a simple and consistent way to run MySQL across development and testing environments.
version: "3.8"
services:
mysql:
image: mysql:8
container_name: mysql
environment:
MYSQL_DATABASE: app_db
MYSQL_USER: app_user
MYSQL_PASSWORD: app_password
MYSQL_ROOT_PASSWORD: root_password
ports:
- "3306:3306"
volumes:
- mysql-data:/var/lib/mysql
volumes:
mysql-data:
This setup ensures persistent storage while exposing MySQL locally for application access.
Minimal, Production-Ready HikariCP Configuration
The following Java configuration demonstrates a clean and effective HikariCP setup using environment variables for configuration.
import com.zaxxer.hikari.HikariConfig;
import com.zaxxer.hikari.HikariDataSource;
import javax.sql.DataSource;
public class DataSourceFactory {
public static DataSource create() {
HikariConfig config = new HikariConfig();
config.setJdbcUrl(System.getenv("JDBC_URL"));
config.setUsername(System.getenv("MYSQL_USER"));
config.setPassword(System.getenv("MYSQL_PASSWORD"));
config.setMaximumPoolSize(20);
config.setMinimumIdle(5);
config.setConnectionTimeout(30_000);
config.setIdleTimeout(300_000);
config.setMaxLifetime(1_800_000);
return new HikariDataSource(config);
}
}
Why These Settings Matter
- Maximum pool size should reflect database capacity, query latency, and transaction duration — not simply CPU core count.
- Idle and lifetime limits help prevent stale or server-terminated connections.
- Connection timeouts provide backpressure when the database is under load.
In larger applications, this factory can be lifecycle-managed by a dependency injection framework or application container.
Scalable Browser Automation with Selenium Grid
The Challenge of UI Test Scaling
Browser-based tests are inherently resource-intensive. Each browser instance consumes significant memory and CPU, making local execution poorly suited for large test suites. As the number of tests grows, execution time and infrastructure costs increase rapidly.
Selenium Grid addresses this problem by enabling parallel execution across multiple browser nodes.
Selenium Grid with Docker Compose
Docker simplifies Selenium Grid deployment by managing the hub and browser nodes as containers.
version: "3.8"
services:
selenium-hub:
image: selenium/hub:4.27.0
ports:
- "4444:4444"
chrome:
image: selenium/node-chrome:4.27.0
depends_on:
- selenium-hub
environment:
SE_EVENT_BUS_HOST: selenium-hub
SE_EVENT_BUS_PUBLISH_PORT: 4442
SE_EVENT_BUS_SUBSCRIBE_PORT: 4443
Scaling note:
When using Docker Compose (non-Swarm), scale browser nodes with:
docker compose up --scale chrome=10
While Selenium Grid enables parallel execution, real-world speedups depend on available memory, test isolation, and external system dependencies.
Connecting a Java Application to Selenium Grid
Once the grid is running, applications can connect using a remote WebDriver.
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.chrome.ChromeOptions;
import org.openqa.selenium.remote.RemoteWebDriver;
import java.net.URL;
public class DriverFactory {
public static WebDriver create() {
ChromeOptions options = new ChromeOptions();
options.addArguments("--headless=new");
options.addArguments("--no-sandbox");
options.addArguments("--disable-dev-shm-usage");
try {
return new RemoteWebDriver(
new URL(System.getenv("REMOTE_WEBDRIVER_URL")),
options
);
} catch (Exception e) {
throw new RuntimeException("Failed to create WebDriver", e);
}
}
}
This approach cleanly separates browser configuration from test logic while allowing Selenium Grid to manage browser allocation and concurrency.
Best Practices for Efficient Resource Usage
Database
- Size connection pools based on database limits, query latency, and transaction duration
- Monitor pool metrics such as active connections and wait times
- Avoid long-running or unbounded transactions
Selenium Grid
- Scale nodes based primarily on available memory
- Always terminate WebDriver sessions explicitly
- Prefer headless browsers in CI environments
Security
- Store credentials in environment variables or secret managers
- Avoid committing sensitive configuration to source control
Conclusion
Efficient resource management is a cornerstone of scalable system design. By combining HikariCP for high-performance database connection pooling and Selenium Grid for parallel browser automation, teams can significantly improve application stability, throughput, and test execution speed.
These tools do not eliminate system limits, but they provide controlled, predictable ways to operate within them. When implemented thoughtfully, they form a strong foundation for backend services and automation pipelines in modern software architectures.