Downloading Files via ECS Exec

Hey, it's aka.

When using ECS Exec, you can't directly download files — so here's how I worked around it. I'll show you a method that combines base64 encoding with redirection. There's also a copy-paste-ready one-liner included.

Uploading files is doable if you're willing to copy-paste your way through...

Prerequisites

• ECS Exec is enabled on your ECS cluster.
  • There are several setup steps, so refer to the official documentation for configuration.
  • https://docs.aws.amazon.com/AmazonECS/latest/developerguide/ecs-exec.html#ecs-exec-enabling-and-using
• You can use ECS Exec from your local machine.
• The base64 command is available on both the remote and local machines.
• (Optional) Compression commands are available on both remote and local machines.
  • I'll use gzip here, but tar is more versatile, so feel free to use that instead.

⚠️ Notes

1. I'm using Mac's sed, so GNU sed may behave differently.
2. For large files, downloading this way can put a strain on the system. In that case, uploading to S3 first is recommended, as it can affect ECS performance.

Conclusion

CLUSTER=ClusterName
CONTAINER=ContainerName
TASK_ID=TaskId
F=FileName.gz.base64.stdout

aws ecs execute-command --cluster $CLUSTER --task $TASK_ID --container $CONTAINER --interactive --command "bash -c \"gzip $TF -c | base64\"" > $F
L=$(wc -l $F| awk '{print $1}')
awk -v L=$L 'NR==6,NR==L-4 {print $1}' $F | sed -E $'s/\\\r?$//g' | awk '{print}' ORS='' > ${F%".stdout"}
base64 -d -i ${F%".stdout"} -o ${F%".base64.stdout"}
gzip -d ${F%".base64.stdout"}

Explanation

What we're doing is quite simple:

1. Compress the file + base64 encode it, then redirect the output to a local file

2. Remove the ECS Exec session messages (lines 1-5 and from 3 lines before the last line to the last line)

   • Example of ECS Exec session messages:

     
     The Session Manager plugin was installed successfully. Use the AWS CLI to start a session.
     
     Starting session with SessionId: ...
     
     ${compressed + base64 encoded file content}
     
     Exiting session with sessionId: ...
     

3. Remove line breaks to concatenate everything into a single line

   1. ECS Exec automatically wraps (inserts line breaks) when a single line exceeds a certain character limit

4. base64 decode + decompress the file

Stripping the last 3 lines and handling the line breaks is a bit tedious, but once you have the commands figured out, it's straightforward.

Summary

In this post, I covered how to download files via ECS Exec.

Apart from the ECS Exec-specific processing, what we're doing is quite simple. It's similar to what general network communication does, after all.

Side Note

Having written all this...

ECS Exec is convenient, but I recommend not using it for the following reasons:

1. [Security] Having ECS Exec enabled is a security hole in itself.
2. [Application Design & Operations] File operations should use S3 or EFS.
3. [Application Design & Operations] For logging and performance monitoring, use external services instead of ECS Exec. Both can be achieved with CloudWatch features.
4. [Application Design & Operations] Even for use cases beyond 2 and 3, if ECS Exec is needed, it means manual operations are occurring — ideally, those operations should be automated.