AWS

API reference for chia.aws. These pages are generated from the docstrings in the source, so they stay in sync with the code.

Config

class chia.aws.config.AWSConfig(region: str = 'us-east-1', key_name: str = 'firesim', vpc_name: str = 'firesim', security_group_name: str = 'for-farms-only-firesim', ssh_user: str = 'ubuntu', ssh_private_key: str | None = None, use_public_ip: bool = False, s3_bucket: str = 'firesim-chia-builds', subnet_id: str | None = None, aws_creds_dir: str | None = None)

Bases: object

AWS infrastructure configuration for FireSim operations.

class chia.aws.config.EC2InstanceConfig(instance_type: str, volume_size_gb: int = 200, market: str = 'ondemand', spot_max_price: str = 'ondemand', spot_interruption_behavior: str = 'terminate', ami_id: str | None = None, tags: dict[str, str]=<factory>, user_data: str | None = None)

Bases: object

Configuration for a single EC2 instance type to launch.

EC2

Thin wrapper around AWS EC2 operations for launching/terminating instances.

Imports FireSim’s awstools functions when available, falling back to direct boto3 calls otherwise.

class chia.aws.ec2.EC2Instance(instance_id: str, private_ip: str, public_ip: str | None, instance_type: str)

Bases: object

Serializable representation of a launched EC2 instance.

chia.aws.ec2.get_default_ami(region: str | None = None) → str

Look up the F2/FPGA Developer AMI (Ubuntu) in the current (or given) region.

Uses the same AMI name pattern as FireSim’s awstools.get_f2_ami_name(), with fallback attempts for hotfix version bumps.

chia.aws.ec2.launch_ec2_instances(aws_config: AWSConfig, instance_config: EC2InstanceConfig, count: int = 1, instance_name: str = 'chia-firesim') → list[EC2Instance]

Launch EC2 instances and return serializable EC2Instance objects.

Parameters:

• aws_config – AWS/VPC configuration.

• instance_config – Instance type and market configuration.

• count – Number of instances to launch.

Returns:

List of EC2Instance dataclasses (safe for Ray serialization).

chia.aws.ec2.wait_for_instances(instance_ids: list[str], region: str = 'us-east-1', timeout: float = 600) → list[EC2Instance]

Wait for instances to reach ‘running’ state and populate IP addresses.

Parameters:

• instance_ids – List of EC2 instance IDs to wait on.

• region – AWS region.

• timeout – Maximum seconds to wait.

Returns:

Updated list of EC2Instance with IP addresses filled in.

chia.aws.ec2.terminate_ec2_instances(instance_ids: list[str], region: str = 'us-east-1', dryrun: bool = False, wait: bool = True, wait_timeout: int = 600) → None

Terminate EC2 instances by ID.

Parameters:

• instance_ids – Instance IDs to terminate.

• region – AWS region.

• dryrun – If True, validate the request without actually terminating.

• wait – If True, block until every instance reaches the terminated state. Ensures the caller can safely assume capacity/quota has been released before returning.

• wait_timeout – Max seconds to wait for termination (when wait=True).

Host

EphemeralEC2Host — combines an EC2 instance with Chia’s SSHClient for remote execution, with automatic cleanup via context manager.

class chia.aws.host.EphemeralEC2Host(ec2_instance: EC2Instance, aws_config: AWSConfig)

Bases: object

An ephemeral EC2 instance paired with an SSH connection.

Intended for use as a context manager so the instance is always terminated, even on exceptions:

with EphemeralEC2Host(ec2_inst, aws_cfg) as host:
    host.wait_ready()
    host.run("echo hello")

wait_ready(timeout: float = 300) → None

Block until SSH is reachable.

rsync_up(local_path: str, remote_path: str, exclude: list[str] | None = None, filter_rules: list[str] | None = None) → None

Upload files from the local machine to this host.

rsync_down(remote_path: str, local_path: str, exclude: list[str] | None = None) → None

Download files from this host to the local machine.

run(cmd: str, timeout: int = 300, check: bool = True, retries: int = 0, retry_delay: float = 5.0)

Run a command over SSH.

run_script(commands: list[str], timeout: int = 600, check: bool = True)

Run multiple commands in a single SSH session.

terminate() → None

Terminate this EC2 instance.

S3

S3 object-store client bound to one bucket, exposed as a chia service-pattern node.

binds to one bucket at construction and exposes a small, synchronous API. Errors are raised as typed exceptions (no in-band success: bool); see S3Error and its subclasses. Transient failures (5xx, throttling, network errors) are retried once before raising.

Usage:

from chia.aws.s3 import S3Node, S3NotFoundError

node = S3Node("my-bucket", region="us-west-2")
node.put_bytes("results/run1.json", b"{}")
data = node.get_bytes("results/run1.json")
node.upload_file("/tmp/waves.vcd", "waves/run1.vcd")

For anything beyond this surface (presigned URLs, multipart tuning, …), drop down to the raw boto3 client via node._client.

exception chia.aws.s3.S3Error(message: str = '', code: str = '')

Bases: Exception

Base class for all S3 node errors. code is the AWS error code, if any.

exception chia.aws.s3.S3AuthError(message: str = '', code: str = '')

Bases: S3Error

Missing/invalid credentials, or access denied (401/403/AccessDenied).

exception chia.aws.s3.S3NotFoundError(message: str = '', code: str = '')

Bases: S3Error

Object or bucket does not exist (404/NoSuchKey/NoSuchBucket).

exception chia.aws.s3.S3RequestError(message: str = '', code: str = '')

Bases: S3Error

Non-transient client-side error (other 4xx, e.g. BucketAlreadyExists).

exception chia.aws.s3.S3ServerError(message: str = '', code: str = '')

Bases: S3Error

5xx/throttling after one retry, or a network/timeout failure.

class chia.aws.s3.S3ObjectInfo(key: str, size: int, last_modified: str, etag: str)

Bases: object

Serializable metadata for one S3 object (one list result row).

class chia.aws.s3.S3Node(bucket: str, region: str | None = None, profile: str | None = None, aws_access_key_id: str | None = None, aws_secret_access_key: str | None = None, aws_session_token: str | None = None, timeout_seconds: int = 60, logging_level: int = 10)

Bases: object

Client for one S3 bucket.

Service-pattern node (head-node only, not a Ray task). Bind to one bucket at construction; all methods are synchronous and raise typed exceptions on failure. Credentials come from the default boto3 chain unless overridden — see __init__ for the resolution order.

Bind to bucket. Credential resolution, in order of precedence:

1. Explicit keys — pass aws_access_key_id and aws_secret_access_key together (plus aws_session_token for temporary/STS credentials). Use this to ship credentials by value into an environment with no ~/.aws or instance role, e.g. a docker worker.

   key, secret, token = load_aws_creds(creds_dir) # on the head node node = S3Node(bucket, # on the worker

   aws_access_key_id=key, aws_secret_access_key=secret, aws_session_token=token or None)

   Empty strings are treated as “not provided” (so a blank load_aws_creds tuple falls through to the default chain), but passing only one of key/secret, a token without both keys, or explicit keys together with profile raises ValueError.

2. profile — a named profile from ~/.aws.

3. Neither — boto3’s default chain (env vars, ~/.aws, instance metadata / IAM role). The right choice on the head node and on EC2.

Credentials are resolved lazily by boto3: nothing is validated here, and missing/invalid credentials surface as S3AuthError on the first call.

upload_file(local_path: str | Path, key: str) → None

Upload a local file to key (multipart for large files).

download_file(key: str, local_path: str | Path) → None

Download key to a local path, creating parent directories.

put_bytes(key: str, data: bytes, content_type: str | None = None) → None

Write data to key.

get_bytes(key: str) → bytes

Return the contents of key.

list(prefix: str = '', max_keys: int | None = None) → list[S3ObjectInfo]

List objects under prefix, up to max_keys (None = all).

Includes zero-byte .../ folder-marker keys if present; callers that don’t want them should filter on key.endswith("/").

exists(key: str) → bool

Return whether key exists. A 403 raises S3AuthError rather than reporting False.

delete(key: str) → None

Delete key. Idempotent — S3 does not error on a missing key.

ensure_bucket() → bool

Create the bound bucket if it does not exist.

Returns True if the bucket was created, False if it already existed. Raises S3AuthError if the bucket exists but is not accessible.