Service execution-role proxy

The malicious API calls are attributed to the workload’s execution role rather than directly to the identity that submitted the code. This makes the attack’s origin harder to establish from downstream activity alone: investigators must connect the role’s requests to the workload submission. The role may also reach data or APIs the submitter cannot access directly. This is an attribution obstacle, not anonymity. The submission event still records the immediate submitting identity, which may be an intermediary when another service submitted the work.^[1][2][3]

Which role runs the code

The examples here concern workloads created with a selected execution role. Where credentials are supplied to the code, its API calls use that role’s effective permissions. SageMaker network isolation is an important exception: the container receives no AWS credentials, while the service uses the role separately for managed S3 input and output.^[1:1][4]

flowchart LR
    A([Submitting identity]) --> W[Managed compute workload]
    R[Passed execution role] -->|Service assumes role| W
    W --> C[Attacker-controlled code]
    C -->|Where container credentials are supplied| T(Temporary role session)
    T --> X([AWS API actions as execution role])
    class A,X principal
    class W,R,C awsResource
    class T credential

Review workload-submission permissions, role-passing permissions, and service trust together. They are separate authorization controls, not interchangeable evidence that the resulting workload is appropriate.^[1:2]

Production data through a training job

A data engineering role can create SageMaker training jobs and pass a shared machine learning (ML) execution role, which can read several production S3 prefixes because existing training pipelines use the same data.

After compromising the data engineering role, the attacker submits a short training job containing code that lists and downloads the production objects. CloudTrail records CreateTrainingJob under the compromised role. Later object reads use the shared execution role’s SageMaker session, but GetObject appears in CloudTrail only when the relevant S3 data events are enabled.^[5]

SageMaker training jobs

In SageMaker, the Python software development kit (SDK) @remote decorator packages local code and dependencies, stages them in S3, and runs the function as a training job.^[6] Without network isolation, code can receive the job’s execution-role credentials. With network isolation enabled, containers receive no AWS credentials and cannot make outbound calls, including to S3. SageMaker still performs service-managed S3 downloads and uploads using the execution role outside the container; these operations must be distinguished from container-originated activity.^[4:1]

import os

import boto3
from sagemaker.core.helper.session_helper import Session
from sagemaker.remote_function import remote


session = Session(
    boto_session=boto3.Session(region_name=os.environ["AWS_REGION"])
)


@remote(
    sagemaker_session=session,
    role=os.environ["SAGEMAKER_ROLE_ARN"],
    instance_type="ml.m5.xlarge",
    dependencies="auto_capture",
)
def effective_identity():
    import boto3

    return boto3.client("sts").get_caller_identity()["Arn"]


print(effective_identity())

Glue Python shell jobs

Glue assumes the supplied IAM role and runs the job’s script from S3.^[7][8]

import json

import boto3


identity = boto3.client("sts").get_caller_identity()
print(json.dumps(identity, default=str))

aws s3 cp identity.py "$SCRIPT_URI"

aws glue create-job \
  --name role-proxy \
  --role "$GLUE_ROLE_ARN" \
  --command "Name=pythonshell,PythonVersion=3.9,ScriptLocation=$SCRIPT_URI" \
  --max-capacity 0.0625

RUN_ID=$(aws glue start-job-run \
  --job-name role-proxy \
  --query JobRunId --output text)

aws glue get-job-run \
  --job-name role-proxy \
  --run-id "$RUN_ID" \
  --query 'JobRun.JobRunState'

CodeBuild projects

CodeBuild also runs submitted instructions under its project service role. Because a project can use NO_SOURCE with an inline buildspec, a caller who can create the project does not need repository access.^[9][10]

{
  "name": "role-proxy",
  "serviceRole": "<CODEBUILD_ROLE_ARN>",
  "source": {
    "type": "NO_SOURCE",
    "buildspec": "version: 0.2\nphases:\n  build:\n    commands:\n      - aws sts get-caller-identity\n"
  },
  "artifacts": {
    "type": "NO_ARTIFACTS"
  },
  "environment": {
    "type": "LINUX_CONTAINER",
    "computeType": "BUILD_GENERAL1_SMALL",
    "image": "aws/codebuild/standard:7.0",
    "imagePullCredentialsType": "CODEBUILD",
    "privilegedMode": false
  }
}

aws codebuild create-project \
  --cli-input-json file://project.json

BUILD_ID=$(aws codebuild start-build \
  --project-name role-proxy \
  --query 'build.id' --output text)

aws codebuild batch-get-builds \
  --ids "$BUILD_ID" \
  --query 'builds[0].{Status:buildStatus,Logs:logs.deepLink}'

Tracing the passed role

Start with the workload submission and investigate candidate downstream activity by the selected role. Look for a new caller-to-role pairing followed by access outside the role’s normal workload, without assuming that a shared role uniquely identifies the submitted job.

A SageMaker submission can record the selected role, training image, isolation setting, and workload identifier. The immediate caller can be a service intermediary, and field presence varies by event:^{[13][14][2:2]}

{
  "eventSource": "sagemaker.amazonaws.com",
  "eventName": "CreateTrainingJob",
  "userIdentity": { "arn": "<submitting identity ARN>" },
  "requestParameters": {
    "trainingJobName": "<job name>",
    "roleArn": "<execution role ARN>",
    "algorithmSpecification": {
      "trainingImage": "<image URI>"
    },
    "enableNetworkIsolation": false
  },
  "responseElements": {
    "trainingJobArn": "<job ARN>"
  }
}

A submission without errorCode records acceptance, not proof that training or build code executed. Confirm runtime state separately.^[15] Later events whose userIdentity.sessionContext.sessionIssuer.arn matches the selected role are candidates, not unique job attribution: concurrent workloads and service-managed operations can use the same role. Keep the workload identifier, code or image location, network configuration, and event time with the match. Require service-specific session or workload evidence before attributing an action to a particular job; no universal job-to-session field is guaranteed. Preserve uncertainty if that evidence is unavailable.^[3:1]

Use event time and observed workload start, state changes, and end time for the initial investigation. If lifecycle evidence is missing, set an initial horizon using local queue and startup delays plus logging latency. Revisit matches when late or out-of-order events arrive, and extend the review for continued workload execution or credible later session activity. Neither the horizon nor job termination bounds exposure from credentials already issued.

Judge the activity against the service’s expected caller, role, and code relationships. An approved pipeline’s usual image or script can explain a familiar submission; a familiar execution role alone should not excuse a new principal passing it to a new workload.

Restricting role selection

• Scope iam:PassRole to exact workload roles and require iam:PassedToService to match the intended service principal.
• Grant CreateTrainingJob, CreateJob, CreateProject, and corresponding update or start actions only to deployment identities that need them.
• Separate execution roles by workload and data boundary. A training role, Glue role, or build role should not carry unrelated administration permissions.
• Restrict SageMaker training images, Glue script prefixes, and CodeBuild source or buildspec changes to controlled repositories and S3 locations.
• Enable SageMaker network isolation where compatible. It removes container credentials and outbound calls, but not the service’s separate execution-role access for managed S3 I/O.^[4:2]
• For Glue and CodeBuild, use appropriately restricted egress and endpoint access where supported. A private subnet alone does not remove outbound access, and endpoint policies govern only requests using that endpoint. These controls do not follow credentials copied to another host.^[16][17]

These deployment and iam:PassRole restrictions limit future unauthorized submissions; they do not neutralize already configured or running workloads. Role passing generally occurs during setup, not every time the service assumes the role.^[1:3] Remove compromised submission access and malicious workload configuration, stop or remediate existing unauthorized workloads, and address further credential issuance separately from denying already-issued sessions.

Applicable policy changes can deny existing temporary credentials after propagation. Role-session revocation targets sessions issued before its cutoff, leaving later sessions outside that denial. Confirm workload containment and effective session denial separately rather than treating an accepted configuration change or completed job as containment.^[18][19]

References