Topic:   aws CLI: why is copying files from EBT worker tier to s3 so extremly slow?
Mar 31, 2021 12:11 1 Replies 147 Views HARSHA

we are running an application in an ElasticBeansTalk Worker Tier. After processing the produced files are copied to an S3 bucket.

everything is working fine so far. However it turned out that copying even small amounts of files to the S3 Bucket is extremly slow

a typical example is 7 files with a total of 200KBytes that takes nearly 2 seconds. Situation improves when the files are larger, e.g. 5 files with a total of 30MB is also around 2 seconds

What is the reason for this? When the same files are copied from my local developer machine (100MBit only) it takes nearly no time. But - as said - from the Worker Tier (5GBit network I think) it is taking so incredible long.

used command ...

aws s3 cp --only-show-errors --recursive out s3://bucket_name/some_sub_folders/out --acl public-read --expires "Wed Mar 31 11:03:39 UTC 2021"

du -h out
---> 212K out

find out

out/sample_1.png
out/sample_2.png
out/sample_3.png
out/sample_4.png
out/workload.json
out/service.log.txt
 

Prev Next
Topic Replies (1)
  1. 1
    idnkx user

    SHALINI

    possible mitigation ...

    explicitly set env var AWS_DEFAULT_REGION to the appropriate region, e.g. eu-central-1

    before: 2.0 seconds (reproducable)
    now: 1.5 Seconds (reproducable)

Leave a Reply
Guest User

Not sure what course is right for you?

Choose the right course for you.
Get the help of our experts and find a course that best suits your needs.


Let`s Connect