Deploying a Secure DICOM Medical Image Classification API on Google Cloud Vertex AI

Why Vertex AI custom containers for DICOM

DICOM files are not plain images — they carry a binary header with patient metadata, imaging parameters, and pixel data that needs correct windowing before it resembles a normal PNG/JPEG a CNN can consume. Vertex AI's prebuilt prediction containers expect flat image tensors, so this pipeline uses a custom prediction container that decodes DICOM, strips identifying metadata, applies windowing, then runs inference — all inside one request so raw DICOM never has to be pre-converted and stored insecurely outside the trust boundary.

Google Cloud Healthcare API and Vertex AI are both covered under Google Cloud's HIPAA BAA (which you must accept in the Google Cloud console before processing PHI) — confirm this and complete your own compliance review before handling real patient studies.

1. Technical architecture

┌────────────┐   HTTPS (mTLS via IAP)   ┌─────────────────────────┐
│ Client (PACS │ ───────────────────────▶│ Cloud Run / API Gateway    │
│ or ordering    │                       │ (auth: IAM + service       │
│ system)        │                       │ account token)              │
└────────────┘                          └───────────┬─────────────┘
                                                       ▼
                                        ┌───────────────────────────┐
                                        │ Vertex AI Endpoint            │
                                        │ (custom prediction container)  │
                                        └───────────┬───────────────┘
                     ┌──────────────────────────────┼───────────────────────────────┐
                     ▼                               ▼                               ▼
        ┌────────────────────┐        ┌────────────────────────┐        ┌───────────────────────┐
        │ 1. DICOM decode &     │  ──▶  │ 2. De-identify tags &     │  ──▶  │ 3. Window + resize →     │
        │    validate (pydicom) │        │    windowing (numpy)       │        │    model inference (TF)   │
        └────────────────────┘        └────────────────────────┘        └──────────┬────────────┘
                                                                                        ▼
                                                                          ┌───────────────────────┐
                                                                          │ Response: class label +  │
                                                                          │ confidence, NO pixel data │
                                                                          │ echoed back                │
                                                                          └───────────────────────┘

Storage (async, optional): de-identified study metadata + prediction logged to
BigQuery for model monitoring; raw pixel data is NOT persisted by the API.

2. Complete code implementation

2.1 Preprocessing module

# predictor/preprocessing.py
import io
import numpy as np
import pydicom
from pydicom.errors import InvalidDicomError

# Tags that must be stripped before anything derived from this file leaves
# the container, per DICOM PS3.15 Annex E "Basic Application Level De-Identification".
PHI_TAGS_TO_REMOVE = [
    (0x0010, 0x0010),  # PatientName
    (0x0010, 0x0020),  # PatientID
    (0x0010, 0x0030),  # PatientBirthDate
    (0x0010, 0x1040),  # PatientAddress
    (0x0008, 0x0090),  # ReferringPhysicianName
    (0x0008, 0x0080),  # InstitutionName
]


def load_dicom(file_bytes: bytes) -> pydicom.Dataset:
    try:
        ds = pydicom.dcmread(io.BytesIO(file_bytes))
    except InvalidDicomError as exc:
        raise ValueError("Uploaded file is not a valid DICOM instance") from exc

    if "PixelData" not in ds:
        raise ValueError("DICOM instance has no PixelData element — cannot classify")

    return ds


def strip_phi(ds: pydicom.Dataset) -> pydicom.Dataset:
    for tag in PHI_TAGS_TO_REMOVE:
        if tag in ds:
            del ds[tag]
    return ds


def apply_windowing(ds: pydicom.Dataset) -> np.ndarray:
    """Converts raw pixel data to an 8-bit array using the DICOM window
    center/width if present, falling back to a full-range min-max stretch."""
    pixels = ds.pixel_array.astype(np.float32)

    center = ds.get("WindowCenter", None)
    width = ds.get("WindowWidth", None)
    if center is not None and width is not None:
        center = float(center[0] if hasattr(center, "__iter__") else center)
        width = float(width[0] if hasattr(width, "__iter__") else width)
        low, high = center - width / 2, center + width / 2
    else:
        low, high = float(pixels.min()), float(pixels.max())

    if high <= low:
        raise ValueError("Invalid window range computed from DICOM metadata (high <= low)")

    clipped = np.clip(pixels, low, high)
    normalized = (clipped - low) / (high - low)
    return (normalized * 255).astype(np.uint8)


def preprocess(file_bytes: bytes, target_size: tuple[int, int] = (224, 224)) -> np.ndarray:
    ds = load_dicom(file_bytes)
    strip_phi(ds)
    image = apply_windowing(ds)

    # Resize without an extra heavyweight dependency
    from PIL import Image
    pil_img = Image.fromarray(image).convert("L").resize(target_size)
    arr = np.array(pil_img, dtype=np.float32) / 255.0
    return np.stack([arr] * 3, axis=-1)  # replicate to 3 channels for a standard CNN input

2.2 Custom prediction container

# predictor/main.py
import os
import base64
import logging
import numpy as np
import tensorflow as tf
from fastapi import FastAPI, Request, HTTPException
from preprocessing import preprocess

logging.basicConfig(level=logging.INFO)
logger = logging.getLogger("dicom-predictor")

app = FastAPI()
MODEL_PATH = os.environ.get("AIP_STORAGE_URI", "/model")
LABELS = ["normal", "nodule_suspicious", "effusion", "consolidation"]

model = tf.keras.models.load_model(MODEL_PATH)


@app.get(os.environ.get("AIP_HEALTH_ROUTE", "/health"))
def health():
    return {"status": "healthy"}


@app.post(os.environ.get("AIP_PREDICT_ROUTE", "/predict"))
async def predict(request: Request):
    body = await request.json()
    instances = body.get("instances", [])
    if not instances:
        raise HTTPException(status_code=400, detail="Request body must include a non-empty 'instances' array")

    predictions = []
    for i, instance in enumerate(instances):
        b64_data = instance.get("dicom_b64")
        if not b64_data:
            predictions.append({"error": "missing 'dicom_b64' field", "index": i})
            continue
        try:
            file_bytes = base64.b64decode(b64_data)
            tensor = preprocess(file_bytes)
        except ValueError as exc:
            logger.warning("Preprocessing failed for instance %d: %s", i, exc)
            predictions.append({"error": str(exc), "index": i})
            continue
        except Exception as exc:  # unexpected decode/format error
            logger.exception("Unexpected preprocessing failure for instance %d", i)
            predictions.append({"error": "internal preprocessing error", "index": i})
            continue

        batch = np.expand_dims(tensor, axis=0)
        probs = model.predict(batch, verbose=0)[0]
        top_idx = int(np.argmax(probs))
        predictions.append({
            "label": LABELS[top_idx],
            "confidence": float(probs[top_idx]),
            "all_scores": {LABELS[j]: float(probs[j]) for j in range(len(LABELS))},
        })

    return {"predictions": predictions}

# predictor/Dockerfile
FROM python:3.11-slim

WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

COPY main.py preprocessing.py ./

ENV AIP_HEALTH_ROUTE=/health
ENV AIP_PREDICT_ROUTE=/predict
ENV AIP_HTTP_PORT=8080

CMD ["uvicorn", "main:app", "--host", "0.0.0.0", "--port", "8080"]

# predictor/requirements.txt
fastapi==0.111.0
uvicorn[standard]==0.30.1
pydicom==2.4.4
numpy==1.26.4
pillow==10.3.0
tensorflow==2.16.1

3. Step-by-step configuration guide

1. Enable APIs and accept the BAA:

   gcloud services enable aiplatform.googleapis.com artifactregistry.googleapis.com
   

   Accept the Google Cloud HIPAA BAA in the console under Compliance settings before uploading any real patient studies.
2. Build and push the container:

   gcloud artifacts repositories create dicom-predictor --repository-format=docker --location=us-central1
   docker build -t us-central1-docker.pkg.dev/PROJECT_ID/dicom-predictor/classifier:v1 predictor/
   docker push us-central1-docker.pkg.dev/PROJECT_ID/dicom-predictor/classifier:v1
   

3. Upload the trained Keras model to a Cloud Storage bucket the service account can read: gsutil cp -r ./saved_model gs://YOUR_BUCKET/dicom-model/.
4. Register the model on Vertex AI:

   gcloud ai models upload \
     --region=us-central1 \
     --display-name=dicom-classifier \
     --container-image-uri=us-central1-docker.pkg.dev/PROJECT_ID/dicom-predictor/classifier:v1 \
     --container-health-route=/health \
     --container-predict-route=/predict \
     --container-ports=8080 \
     --artifact-uri=gs://YOUR_BUCKET/dicom-model/
   

5. Create an endpoint and deploy, with private access only:

   gcloud ai endpoints create --region=us-central1 --display-name=dicom-endpoint
   gcloud ai endpoints deploy-model ENDPOINT_ID \
     --region=us-central1 --model=MODEL_ID \
     --display-name=dicom-classifier-v1 \
     --machine-type=n1-standard-4 \
     --min-replica-count=1 --max-replica-count=3
   

6. Restrict IAM. Grant only roles/aiplatform.user to the specific service account your ordering system uses to call this endpoint — never grant it to allUsers or allAuthenticatedUsers.
7. Test with a sample (de-identified) DICOM file:

   import base64, json
   from google.cloud import aiplatform
   
   aiplatform.init(project="PROJECT_ID", location="us-central1")
   endpoint = aiplatform.Endpoint("ENDPOINT_ID")
   with open("sample.dcm", "rb") as f:
       b64 = base64.b64encode(f.read()).decode()
   result = endpoint.predict(instances=[{"dicom_b64": b64}])
   print(result.predictions)
   

4. Error handling and edge cases

• Invalid or corrupted DICOM files raise InvalidDicomError, caught in load_dicom and converted into a clean per-instance error entry rather than crashing the whole batch request — one bad file in a batch shouldn't fail every other study.
• Missing PixelData (e.g., a DICOM structured report with no image) is explicitly checked and rejected with a descriptive error instead of failing deep inside apply_windowing.
• Zero or inverted window range (high <= low) is guarded against explicitly — a malformed WindowCenter/WindowWidth pair would otherwise silently divide by zero or produce an all-black image that the model confidently misclassifies.
• Batch partial failures are isolated per-instance in predict() — a malformed file at index 3 of a 10-image batch returns an error object at index 3 while the other 9 predictions still succeed.
• Endpoint autoscaling cold starts — TensorFlow model load time can push cold-start latency past client timeouts; set --min-replica-count=1 (as above) rather than 0 for latency-sensitive clinical workflows, accepting the always-on cost.
• Access token expiration on the client side (401/403 from the endpoint) means the caller's Google Cloud auth token needs refreshing — use google.auth.transport.requests.Request() to auto-refresh credentials rather than caching a token indefinitely.
• PHI leakage in error messages — the except Exception branch in predict() deliberately logs full details server-side via logger.exception but returns only a generic "internal preprocessing error" string to the caller, so stack traces (which could echo file paths or metadata) never leave the container.
• Model/label drift — log label, confidence, and a study hash (not the image itself) to BigQuery for ongoing monitoring, and alert if the distribution of predicted labels shifts sharply from your validation baseline, which often signals an upstream imaging protocol change rather than a real prevalence shift.