← Blog
Containerizing a Low-Code iPaaS for CI/CD

Containerizing a Low-Code iPaaS for CI/CD

dockerci-cdcontainerizationdevopslow-code

In Part 1, I wrote about building a CI pipeline for the platform. a low-code platform that wasn’t designed for automated testing. We got it working, but the biggest remaining bottleneck was environment management. Each platform environment was a heavyweight, stateful deployment that took significant effort to provision and clean up. Builds ran serially. Environment drift was a constant threat.

The solution was containerization. In 2017, that meant Docker. and it meant figuring out how to stuff a distributed enterprise platform into containers when most people were still containerizing stateless web apps.

Why Containerization Was Non-Negotiable

Our CI pipeline worked, but it had a fundamental problem: environments were snowflakes. Every CI run left artifacts behind. database state, cached objects, temporary files, configuration drift. Even with cleanup scripts, each run started from a slightly different baseline than the last.

We needed:

  • Disposable environments. spin up clean, run tests, tear down. Every time.
  • Parallel builds. stop serializing CI runs because we only had one environment.
  • Developer environments. let devs run a local platform instance instead of fighting over shared servers.
  • Reproducibility. same inputs, same environment, same results.

Containerization was the only path that addressed all four.

Understanding What We Were Containerizing

The platform isn’t a single process. It’s a distributed system with several components that need to work together:

  • Application Server. The core runtime (JBoss/Tomcat-based) that executes process models, serves the UI, and runs the engine
  • Search Server. An embedded search engine (based on Elasticsearch at the time) for full-text search and indexing
  • Data Server. An in-memory data grid for caching and real-time analytics
  • Database. Relational database (MySQL in our case) storing application data, process history, and platform metadata

These components communicate over specific ports, expect shared filesystems for certain operations, and have startup order dependencies. This wasn’t “put a Dockerfile on your Express app” territory.

The Container Architecture

Here’s what we ended up with:

graph TB
    subgraph Docker Compose Stack
        subgraph platform-engine [Platform Engine Container]
            AS[Application Server<br/>JBoss + Platform WAR]
            SS[Search Server]
            DS[Data Server]
        end
        
        DB[(MySQL Container)]
        
        subgraph volumes [Shared Volumes]
            V1[platform-data<br/>Application files]
            V2[platform-logs<br/>Log output]
            V3[mysql-data<br/>Database persistence]
        end
    end
    
    AS --> DB
    AS --> SS
    AS --> DS
    
    AS -.-> V1
    AS -.-> V2
    DB -.-> V3
    
    CI[CI Pipeline<br/>Jenkins] --> |docker-compose up| platform-engine
    CI --> |Liquibase| DB
    CI --> |Import package| AS
    CI --> |Run tests| AS
    CI --> |docker-compose down| platform-engine

We consolidated the platform components into a single container rather than splitting them into separate services. Purists would object. and they’d be right in principle. but the inter-component communication was so tightly coupled that separating them created more problems than it solved. The components expected to share memory space and filesystem paths. Fighting that wasn’t worth the architectural purity.

MySQL ran in its own container because it genuinely was an independent service with a clean network boundary.

Building the Images

The Platform Container

The platform Dockerfile was the hardest part. The platform’s installation process was designed for bare-metal or VM deployments. It expected an interactive installer, specific directory structures, and pre-configured system settings.

We reverse-engineered the installer to understand what it actually did, then replicated those steps in a Dockerfile:

FROM centos:7

# System dependencies the platform expects
RUN yum install -y java-1.8.0-openjdk-devel \
    wget unzip \
    && yum clean all

# Create the platform directory structure
RUN mkdir -p /opt/platform/ae \
    /opt/platform/search-server \
    /opt/platform/data-server \
    /opt/platform/logs

# Copy pre-extracted platform installation
COPY platform-installation/ /opt/platform/

# Configuration templates (envsubst at runtime)
COPY config-templates/ /opt/platform/config-templates/

# Entrypoint handles config generation and startup ordering
COPY entrypoint.sh /entrypoint.sh
RUN chmod +x /entrypoint.sh

EXPOSE 8080 8443 2181 5701

ENTRYPOINT ["/entrypoint.sh"]

The real complexity lived in entrypoint.sh. This script had to:

  1. Generate configuration files from environment variables (database connection strings, memory settings, ports)
  2. Start the search server and wait for it to be healthy
  3. Start the data server and wait for cluster formation
  4. Start the application server
  5. Wait for the platform to finish its internal initialization
  6. Signal readiness

Each “wait for” step needed its own health check because the components didn’t start instantly, and starting the next one too early would cause cascading failures.

Startup Ordering

This was the single most frustrating aspect. The platform’s components have implicit startup dependencies that aren’t documented. We discovered them the hard way:

  1. Search server must be accepting connections before the app server starts, or the app server throws exceptions and enters a degraded state
  2. Data server must complete its cluster initialization, even as a single-node “cluster”
  3. Database must have completed the platform’s schema initialization (first boot only)
  4. Application server performs its own multi-phase startup. accepting HTTP connections doesn’t mean it’s ready to serve the application

We wrote health check scripts for each phase and chained them with wait-for-it style loops. The full cold-start took about 3-4 minutes, which felt like an eternity compared to modern container startups but was dramatically faster than provisioning a new VM.

The MySQL Container

This was straightforward. we used the stock MySQL image with:

mysql:
  image: mysql:5.7
  environment:
    MYSQL_ROOT_PASSWORD: "${DB_ROOT_PASS}"
    MYSQL_DATABASE: platform
  volumes:
    - mysql-data:/var/lib/mysql
    - ./init-scripts:/docker-entrypoint-initdb.d

The init-scripts directory contained the base schema that the platform expected. On first boot, MySQL would initialize with this schema, and then the platform’s own startup process would run any additional DDL it needed.

Integration With the CI Pipeline

With the platform containerized, the CI pipeline transformed:

# Start clean environment
docker-compose up -d
./wait-for-platform.sh  # Poll until fully healthy

# Run Liquibase migrations
liquibase --url=jdbc:mysql://localhost:3306/platform update

# Import the platform application package
./import-package.sh app-package.zip

# Wait for import to complete
./wait-for-sync.sh

# Run tests
./run-expression-tests.sh
./run-smoke-tests.sh
./run-integration-tests.sh

# Tear down. completely clean
docker-compose down -v  # -v removes volumes too

That -v flag was the whole point. Every CI run started from absolute zero. No drift, no leftover state, no “works on the CI server” because someone manually tweaked something.

Parallel Builds

With containerized environments, we could finally run builds in parallel. Each Jenkins executor got its own Docker Compose stack on a different port range:

# docker-compose.ci.yml. port offset via environment variable
services:
  platform:
    ports:
      - "${PORT_OFFSET:-0}80:8080"
  mysql:
    ports:
      - "${PORT_OFFSET:-0}306:3306"

Two concurrent builds meant two isolated platform instances, each with their own database, their own state, their own everything. Build times didn’t change, but throughput doubled overnight.

The Challenges

Image Size

The platform container image was enormous. over 4GB. The platform installation itself was large, plus JDK, plus all the system dependencies. We couldn’t do much about this; multi-stage builds helped trim some fat, but the core platform was just big.

We mitigated this with a local Docker registry that cached the base image. Pulling 4GB from a local registry over gigabit ethernet was fast enough. Pulling from a remote registry was not.

Memory Requirements

The platform is memory-hungry. The application server alone wanted 4-8GB of heap. Add the search server, data server, and MySQL, and a CI environment needed 12-16GB of RAM minimum. This limited how many parallel builds we could run on a single host.

We ended up with dedicated CI hosts. beefy machines with 64GB RAM that could run 3-4 parallel environments. Not cheap, but the time savings on serialized builds justified the hardware cost.

Licensing

I won’t get into specifics here, but containerizing enterprise software raises licensing questions. The platform’s licensing model at the time was based on named environments. We had to work with our vendor account team to ensure our container-based CI environments were properly licensed. This is a conversation you should have early. don’t surprise your vendor.

Persistence and State

For CI, we wanted ephemeral environments. But for development, people wanted persistence. start working, stop for the day, resume tomorrow. We supported both modes:

  • CI mode: docker-compose down -v after every run
  • Dev mode: docker-compose stop / docker-compose start to pause and resume

Named volumes made dev mode work. Developers could blow away their environment and start fresh whenever they wanted, or keep it running for days.

What It Enabled

Once the containerized platform was stable, things accelerated quickly:

Feature branch environments. Want to test a feature branch? Spin up a container stack, import the branch’s package, test it, tear it down. This was previously a multi-day request to the infrastructure team.

Onboarding. New developers went from “wait two weeks for an environment” to “run docker-compose up and grab coffee.”

Release confidence. With clean-room CI runs, we stopped finding bugs that existed only because of environment state. If tests passed in CI, they passed in production.

Faster feedback. From commit to test results dropped from “whenever the shared CI environment is free” to about 15 minutes. Not blazing by today’s standards, but transformative at the time.

Reflections

This was 2017. Docker was version 17.x. Docker Compose was still relatively new. Kubernetes existed but wasn’t mainstream yet. We were figuring a lot of this out from first principles because there weren’t many examples of containerizing complex enterprise platforms.

Looking back, some things I’d do differently:

Better health checks. Our startup scripts were brittle shell scripts with hardcoded timeouts. Today I’d use Docker’s native HEALTHCHECK instruction and proper readiness probes.

Smaller images. We didn’t invest enough in image optimization. Layer caching, multi-stage builds, and Alpine-based images could have cut our image size significantly.

Infrastructure as code. Our Docker Compose files and scripts lived in a repo, but the CI host configuration was manual. Terraform or Ansible for the CI infrastructure would have made scaling easier.

Don’t fight the monolith. We briefly tried splitting the platform’s components into separate containers. It was the wrong call for this platform. Recognizing when microservice patterns don’t apply. even in a container world. would have saved us a week of dead ends.

The Bigger Lesson

The real takeaway isn’t about Docker or the platform specifically. It’s about refusing to accept platform limitations as permanent constraints. “This platform doesn’t support CI/CD” isn’t a reason to not do CI/CD. it’s a problem to solve.

Low-code platforms are increasingly common. They promise speed and accessibility, and they deliver on those promises. But they often lag behind on engineering practices that traditional development takes for granted. The teams that bridge this gap. that bring CI/CD discipline to low-code environments. ship better software, faster.

It’s not easy. It requires creativity, stubbornness, and a willingness to build tooling the platform vendor hasn’t built yet. But it’s worth it.